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Abstract 

We consider the problem of maximizing a non-negative submodular set function / : 2^ — >■ R+ over a ground set 
subject to a variety of packing type constraints including (multiple) matroid constraints, knapsack constraints, and 
their intersections. In this paper we develop a general framework that allows us to derive a number of new results, in 
particular when / may be a non-monotone function. Our algorithms are based on (approximately) maximizing the 
multilinear extension F of / (6) over a polytope P that represents the constraints, and then effectively rounding the 
fractional solution. Although this approach has been used quite successfully (7] [33] [35] [15] ID, it has been limited in 
some important ways. We overcome these limitations as follows. 

First, we give constant factor approximation algorithms to maximize F over an arbitrary down-closed polytope 
P that has an efficient separation oracle. Previously this was known only for monotone functions 1481 . For non- 
monotone functions, a constant factor was known only when the polytope was either the intersection of a fixed 
number of knapsack constraints 1351 or a matroid polytope 0491 1421 . Second, we show that contention resolution 
schemes are an effective way to round a fractional solution, even when / is non-monotone. In particular, contention 
resolution schemes for different polytopes can be combined to handle the intersection of different constraints. Via LP 
duality we show that a contention resolution scheme for a constraint is related to the correlation gap |2] of weighted 
rank functions of the constraint. This leads to an optimal contention resolution scheme for the matroid polytope. 

Our results provide a broadly applicable framework for maximizing linear and submodular functions subject 
to independence constraints. We give several illustrative examples. Contention resolution schemes may find other 
applications. 

1 Introduction 

We consider the meta-problem of maximizing a non-negative submodular set function subject to independence con- 
straints. Formally, let iV be a finite ground set of cardinality n, and let / : 2^ — )• be a submodular set function 
over Let I C 2^ be a downward-closed family!^ of subsets of N. Our problem is then laaxsei fiS)- We are 
interested in independence families induced by natural and useful constraints such as matroid constraints, knapsack 
constraints, related special cases, and their intersections. Throughout this paper we assume that / is given via a value 
oracle; that is, given a set 5* C the oracle returns f{S). The function / could be monotone or non-monoton^; 
monotone functions typically allow better approximation results. 

Submodular function maximization has recently attracted considerable attention in theoretical computer science. 
This is for a variety of reasons, including diverse applications — a prominent application field being algorithmic game 

* A preliminary version of this paper appeared in Proc. of ACM STOC, 20 11 . 

tDept. of Computer Science, Univ. of Illinois, Urbana, IL 61801. Partially supported by NSF grants CCF-0728782 and CCF-1016684. E-mail: 

chekuri@cs . Illinois . edu. 

tjBM Almaden Research Center, San Jose, CA 95120. E-mail: jvondrak@us . ibm. com. 

§Dept. of Mathematics, MIT, Cambridge, MA 02139. E-mail: ricoz@math .mit . edu. Supported by Swiss National Science Foundation 
grant PBEZP2- 129524, by NSF grants CCF-1 115849 and CCF-0829878, and by ONR grants NO0O14-12-1-O033, N00014-1 1-1-0053 and N00014- 
09-1-0326. 

'A set function / : 2^ ^ R is submodular iff f{A) + f(B) > f{A U B) + f{A n B) for all A,B CN. 
family of sets I C 2^ is downward-closed if for any AgBC_N,BgX implies that A G I. 
is monotone if f{A) < f(B) whenever A C B. 



1 



theory, where submodular functions are very commonly used as utihty functions to describe diminishing returns — and 
also the recognition of interesting algorithmic and structural properties. A number of well-known problems can be 
seen as special cases of submodular function maximization. For example, the APX-hard Max-Cut problem can be 
seen as (unconstrained) maximization of the cut function f : 2^ R_|_ of a graph G = {V,E). (Note that / here is 
non-monotone.) Another well-known special case of our problem is the Max-fc-Cover problem, which can be viewed 
as max{/(S') : \S\ < k} where f{S) = \ Ujes \ is the coverage function for a collection of sets {Ai}. Max-fc- 
Cover is hard to approximate to within a factor of (1 — 1/e + e) for any fixed e > 0, unless P = NP ll20l . Hence we 
focus on approximation algorithm^. 

Classical work in submodular function maximization was based on combinatorial techniques such as the greedy 
algorithm and local search. We mention the work of Comuejols, Fisher, Nemhauser and Wolsey ifTS] 1411 [25] |40l 
from the late 70's which showed a variety of approximation bounds when / is monotone submodular and X is the 
intersection of matroid constraints. Recent algorithmic work has considerably extended and improved the classical 
results. Local-search methods have been identified as particularly useful, especially for non-monotone functions. 
Some of the recent results include the first constant factor approximation for the unconstrained submodular function 
maximization problem 11211 , and a variety of approximation results for knapsack and matroid constraints ||35] |36l . The 
greedy algorithm has also been modified and made applicable to non-monotone functions 1291 . 

Despite the above-mentioned results, combinatorial techniques have some limitations: (i) they have not been able 
to achieve optimal approximation results, except in the basic case of a single cardinality or knapsack constraint ATI l45l ; 
(ii) they do not provide the flexibility to combine constraints of different types. A new approach which overcomes 
some of these obstacles and brings submodular function maximization closer to the world of polyhedral techniques is 
via the multilinear relaxation, introduced in this context in [61. 

A relaxation and rounding framework based on tlie multilinear relaxation. In this paper we introduce a gen- 
eral relaxation and rounding framework for maximizing submodular functions, which builds upon, and significantly 
extends, previous approaches. When dealing with linear (or convex) objective functions, a standard paradigm is to 
design a linear or convex relaxation whose solution is then rounded via a problem-specific procedure. A difficulty 
faced in extending this approach to maximizing a submodular function / : 2^ — > M.^^ — which we often interpret as a 
function on the vertices of a {0, 1}^ hypercube that correspond to incidence vectors — is to find a suitable extension 
g : [0, 1]^ — > R+ of / to the full hypercube. The goal is to leverage such an extension g as follows. Suppose we 
have a polytope Px C [0, 1]^ that is a relaxation for I C 2^ in the sense that {1/ | / € 1} C Pi- We want to 
approximately maximize the continuous problem maxxep^ .9(^) 1° ^^1^ ^ fractional solution x* G Px that is finally 
rounded to a feasible integral solution. 

The best-studied extension of a submodular function is the Lovasz extension 1381 ; however, being a convex func- 
tion, it is mostly suitable for submodular function minimization problems. For maximization of submodular functions, 
the following multilinear extension was introduced in j6], inspired by the work in |IT]: 

scN les j^s 

The value F(x) is equivalently the expected value of f{R) where i? is a random set obtained by picking each element 
i independently with probability Xi. We observe that if / is modulaJI then F is simply a linear function. In this paper 
we focus on the multilinear extension. The two obvious questions that arise when trying to build a general relaxation 
and rounding framework based on the multilinear extension are the following. First, can we (approximately) solve the 
problem maxxePx ^(^)? This question is particularly interesting due to the fact that the multilinear extension is in 
general not concave (nor convex). Second, can we round a fractional solution effectively? 

Recent work has addressed the above questions in several ways. First, Vondrak ||48l gave a continuous greedy 
algorithm that gives an optimal (1 — l/e)-approximation for the problem maxxepF(x) when / is monotone sub- 
modular and P is a solvable polytop^^. When / is non-monotone, the picture is less satisfactory. Lee et al. |35] gave 
a local-search based algorithm that gives a (1/4 — e)-approximation to maximize F over the polytope induced by a 

"*If / is not assumed to be non-negative, even tlie unconstrained problem is inapproximable since deciding whether the optimum value is positive 
or zero requires an exponential number of queries. 

'A function is modular if /(A) + /(B) = f{AUB) + f{AnB) for all A, B C TV. If /is modular then f[A) = m + T.isA'^i for some 
weight function ui : Af — > R. 

^We say that a polytope P is solvable if one can do efficient linear optimization over P. 
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fixed number of knapsack constraints. Vondrak P9l obtained a 0.309-approximation for maximizing F over a single 
matroid polytope, and this ratio has been recently improved to 0.325 However, no approximation algorithm was 
known to maximize F over a general solvable polytope P. 

In terms of rounding a fractional solution x, a natural strategy to preserve the value of F(jx.) in expectation is to 
independently round each coordinate i up to 1 with probability Xi and down to otherwise. However, this rounding 
strategy does not typically preserve the constraints imposed by I. Various dependent rounding schemes have been 
proposed. It was shown in @ that "pipage rounding" can be used to round solutions in the matroid polytope with- 
out losing in terms of the objective function F{x.) (JTSj achieves the same via "swap-rounding"). In ||33l[35l l4l [34l , 
randomized rounding coupled with alteration was used for knapsack constraints. More recently, ifTSi showed concen- 
tration properties for rounding in a single matroid polytope when / is monotone, and lISTl showed concentration for 
independent rounding even when / is non-monotone. These led to a few additional results. Despite this progress, the 
"integrality gap" of max{i^(x) : x e P} has been so far unknown even when / is monotone and P the intersection 
of two matroid polytopes. (We remark that for intersections of matroids, combinatorial algorithms are known to yield 
good approximations 1351 l36l .) However, even for modular functions — i.e., classical linear optimization — combining 
constraints such as matroids and knapsack constraints has been difficult, and no general result was known that matched 
the best bounds one can get for them separately. 

In summary, previous results via the multilinear relaxation were known only for rather restricted cases, both in 
terms of approximately maximizing the multilinear extension, and in terms of effectively rounding fractional solutions. 
We next describe the contributions of this paper in this context. 

Our contribution at a higli level: In this paper we develop a general framework for solving submodular maximization 

problems of the form max{/(S') : S G I}, where f : 2^ ^ M-|_ is submodular and I C 2'^ is a downward-closed 
family of sets. Our framework consists of the following components. 

• Optimizing the multilinear relaxation: We give the first constant factor approximation for the problem max{F(x) : 
X e P} where F is the multilinear extension of any non-negative submodular function, and P is any down- 
monoton^ solvable polytope. 

• Dependent randomized rounding: We propose a general (dependent) randomized rounding framework for mod- 
ular and submodular functions under independence constraints via what we call contention resolution schemes 
(CR schemes). Rounding an approximate maximizer of the relaxation max{F(x) : x e P} via a CR scheme 
that is tailored to the given constraints, leads to a solution with provable approximation guarantee. A key advan- 
tage of CR schemes is the ability to easily combine CR schemes designed for different constraints into a CR 
scheme for the intersection of these constraints. 

• Contention resolution schemes: We present CR schemes for a variety of packing constraints, including knapsack 
constraints, matroid constraints, sparse packing systems, column-restricted packing constraints, and constraints 
imposed by an unsplittable flow problem in paths and trees. Our CR scheme for the matroid polytope, which is 
provably optimal, is obtained by exploiting a tight connection between CR schemes and the correlation gap |2l 
of the associated weighted rank functions. Previously, in the context of matroids, an optimal CR scheme was 
only known for the uniform matroid of rank 1 Il22l |23l . 

The above ingredients can be put together to obtain a relaxation and rounding framework leading to a variety of 
new results that we discuss in more detail in Section |2] We summarize some of our results in Table [T] 

1.1 Maximizing the multilinear extension over a general polytope 

We now give a more detailed description of our technical results and the general framework. First, we give a con- 
stant factor approximation for the problem max{F(x) : x € P}, where F is the multilinear extension of a non- 
monotone submodular function / and P is a down-monotone solvable polytope; the monotone case admits a (1 — 1 /e)- 
approximation 1481 as we mentioned already. The condition of down-monotonicity of the polytope is necessary for 

polytope PC [0, l]'^ is down-monotone if for all x, y € [0, l]'^. y < x and x £ P implies y G P. 
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Constraint type 


Linear maximization 


Monotone submod. max. 


Non-negative submod. max. 


0(1) knapsacks 


[l-e|8 261] 


[l-l/e-e|34|] 


0.325 [0.25 |35|] 


k matroids &i I — 0(1) knapsacks 


T [«((ITl) I28JI29J] 


^ [n(j^) (28JI29J] 


^ [n(^) 12811291] 


fc-matchoid & ^-sparse PIP 








Unsplittable flow in paths and trees 




nil) 


0(1) 



Table 1 : Approximation ratios for different types of constraints and objective functions. Results in square brackets were previously 
known. 

the non-monotone case; it follows from B9ll50l that no constant factor approximation is possible for the matroid base 
polytope which is not down-monotone. 

The main algorithmic technique for non-monotone functions has been local search. Fractional local search with 
additional ideas has been the tool to solve the continuous problem in special cases of poly topes 1351 |49l l42l . Previous 
fractional local search methods ( ||35l and P9l ) improved a current solution x by considering moves along a small 
number of coordinates of x. The analysis took advantage of the combinatorial structure of the underlying constraint 
(knapsacks or matroids) which was sufficiently simple that swaps along a few coordinates sufficed. How do we obtain 
an algorithm that works for any polytope P? 

A new insight: Our key high-level idea is simple yet insightful. Any point x E P can be written as a convex 
combination of the vertices of P. We view the problem of max{P(x) ; x e P} as optimizing a submodular function 
over the ground set consisting of the (exponentially many) vertices of P (duplicated many times in the limit). From 
this viewpoint we obtain a new fractional local search procedure: given a current point x, a local swap corresponds to 
removing a vertex in the convex combination of x and adding a new vertex of P (with appropriate scalar multipliers). 
To implement this efficiently we can use linear optimization over P. (We remark that the continuous greedy algorithm 
for the monotone case |!48l can also be interpreted with this insight.) 

Our algorithms are derived using the above high-level idea. We note that when specialized to the matroid polytope 
or knapsack polytope which have combinatorial structure, our algorithms become simpler and in fact resemble previ- 
ous algorithms. Our algorithms and proofs of approximation guarantees are in fact simpler than the previously given 
proofs for particular poly topes ll35l |49] l42l . 

We present two algorithms following this idea. The first algorithm is close in spirit to the local-search algorithm 
of Lee et al. for knapsack constraints ll35l and gives a 0.25-approximation. This algorithm, despite having a worse 
approximation guarantee then the second one we present, allows us to further explain and formalize the above high- 
level idea in a clean way. The second algorithm uses some ideas of 1491 for the case of a matroid polytope and gives a 
0.309-approximation with respect to the best integer solution in P. 

We would like to mention that subsequently to the conference version of this paper, Feldman et al. [24] presented 
an improved algorithm to maximize the multilinear extension, leading to an (e~^ — e) ~ 0.367-approximation with 
respect to the best integer solution. Their algorithm is an adaptation of the continuous greedy algorithm |48'|. The 
conference version of this paper |[T6) contained a third and much more involved algorithm (generalizing the simulated 
annealing approach of iBSl ) that gives a 0.325-approximation, again with respect to the best integer solution in P. 
For conciseness, and in view of the recent results in (24), we do not include this third algorithm in this paper, and 
concentrate on to two above-mentioned algorithms, which allow us to demonstrate the main new algorithmic insights. 
We summarize our results in the following theorem. 

Theorem 1.1. For any nonnegative submodular function f and a solvable down-monotone polytope P, there is a 0.25- 
approximation algorithmfor the problem max{P(x) : x G P} where F is the multilinear extension of f. There is also 
an algorithmfor this problem which returns a solution y £ P of value F{y) > 0.309 • max{P(x) : x g PH {0, 1}^}- 

We remark that a known limit on the approximability of max{P(x) : x G P} is an information theoretic hardness 
of 0.478-approximationin the value oracle model, even in the special case of a matroid polytope l42l . 
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1.2 Contention resolution schemes 



We show that a certain natural class of rounding schemes that we call contention resolution schemes (CR schemes) 
provides a useful and general framework for rounding fractional solutions under submodular objective functions. For 
a ground set N , let Pi be a convex relaxation of the constraints imposed by I C 2^, and let x e Pi. From 
the definition of F, a natural strategy to round a point x is to independently round the coordinates; however, this 
is unlikely to preserve the constraints imposed by I. Let i?(x) C iV be a random set obtained by including each 
element i ^ N independently with probability Xi. The set i?(x) is not necessarily feasible. We would like to remove 
(randomly) some elements from R{x.), so that we obtain a feasible set / C i?(x). The property we would like to 
achieve is that every element i appears in / with probability at least cXi for some parameter c > 0. We call such 
a scheme "c-balanced contention resolution" for Pi. We stress that a c-balanced CR scheme needs to work for all 
X e Pi. However, often, stronger schemes — i.e. with larger values for c — can be obtained if they only need to work 
for all points in a scaled-down version bPi = {6 • x | x € Pi} of Pi, where b E [0, 1]. Such schemes, which we 
call (6, c)-balanced schemes, will prove to be useful when combining CR schemes for different constraints as we will 
discuss in Section [T3] Below is a formal definition of CR schemes. Let support(x) ^ {i E N \ Xi > 0}. 

Definition 1.2. For 6, c e [0, 1], a (6, c)-balanced CR scheme n for Pi is a procedure that for every x G bPi and 
A C_ N, returns a random set n^iA) satisfying 

(i) 7rx(j4) C j4 n support{x.) and Tr^iA) G I with probability 1, and 

(ii) for all i G support{x.), Pr[i G 7rx(i?(x)) | i G i?(x)] > c. 

The scheme is said to be monotone ifPi'[i G 7rx(Ai)] > Pr[i G ttx{A2)] whenever i E Ai C A2. A (1, c)-balanced 
CR scheme is also called a c-balanced CR scheme. The scheme is deterministic ;/ tt is a deterministic algorithm 
( hence tTx ( A) is a single set instead of a distribution ). It is oblivious ;/ tt is deterministic and tTx (A) — 7Ty{A) for all 
X, y and A, that is, the output is independent of x. and only depends on A. The scheme is efficiently implementable ;/ 
TT is a polynomial -time algorithm that given x, A outputs 7rx(A). 

We emphasize that a CR scheme is defined with respect to a specific polyhedral relaxation Pi of I. Note that on 
the left-hand side of condition ^ for a CR scheme, the probability is with respect to two random sources: first the 
set i?(x) is a random set, and second, the procedure tTx is typically randomized. We note that a (6, c)-balanced CR 
scheme tt can easily be transformed into a fee-balanced CR scheme; details are given in Section |4] 

The theorem below highlights the utility of of CR schemes; when rounding via monotone contention resolution 
schemes, one can claim an expectation bound for submodular functions. A similar theorem was shown in ID for 
monotone functions. We state and prove ours in a form suitable for our context. 

Theorem 1.3. Let f : 2^ M_|_ be a non-negative submodular function with multilinear relaxation F, and x be 
a point in Pi, a convex relaxation for X C 2^. Let tt be a monotone (b, c)-balanced CR scheme for Pi, and let 
I ~ 7rx(^(x)). If f is monotone then 

E[/(/)] >cF(x). 

Furthermore, there is a function 77/ : 2^ — > 2^ that depends on f and can be evaluated in linear time, such that even 
for f non-monotone 

E[/(?y/(/))] >c^^(x). 

As we will see in Section|4l the function 77/ can be chosen to always return a subset of its argument. We therefore 
call it a pruning operation. 

We observe that several previous rounding procedures for packing (and also covering) problems rely on the well- 
known technique of alteration of a set obtained via independent rounding and are examples of CR schemes (see 
ll44l l5ll9l[T4ll4l). However, these schemes are typically oblivious in that they do not depend on x itself (other than 
in picking the random set R), and the alteration is deterministic. Our definition is inspired by the "fair contention 
resolution scheme" in 12211231 which considered the special case of contention for a single item. The dependence on x 
as well as randomization is necessary (even in this case) if we want to obtain an optimal scheme. One key question to 
consider is whether some given down-monotone polytope Pi admits a good (6, c)-balanced CR scheme in the sense 
of small constant values for 6, c. As we will show, many natural constraint systems admit good (6, c)-balanced CR 
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schemes, including matroid constraints, knapsack constraints, and a variety of packing integer programs. In particular, 
to deal with the rather general class of matroid constraints, we exploit a close connection between the existence of CR 
schemes and a recently introduced concept, called correlation gap ||52l . 

Contention resolution via correlation gap and an optimal scheme for matroids: Until recently there was no 
contention resolution scheme for the matroid polytope; an optimal (&, | — )-balanced scheme was previously known 
for the very special case of the uniform matroid of rank one ll22l|23l . We note that the recent work of Chawla et al. 
lflTl[T2l implicitly contains a (&, 1 — &)-balanced deterministic scheme for matroids; their motivation for considering 
this notion was mechanism design. In this paper we develop an optimal scheme for an arbitrary matroicfl 

Theorem 1.4. There is an optimal (6, ^—^ — )-balanced contention resolution scheme for any matroid polytope. More- 
over the scheme is monotone and efficiently implementable. 

The main idea in proving the preceding theorem is consider a randomized CR scheme and view it abstractly as 
a convex combination of deterministic CR schemes. This allows, via LP duality, to show that the best contention 
resolution scheme for a constraint system is related to the notion of correlation gap for weighted rank functions of 
the underlying constraint. We reiterate that the scheme depends on the fractional solution x that we wish to round; 
the alteration of the random set i?(x) is itself a randomized procedure that is tailored to x, and is found by solving a 
linear program. We are inspired to make the general connection to correlation gap due to the recent work of Yan [i52i ; 
he applied a similar idea in the context of greedy posted-price ordering schemes for Bayesian mechanism design, 
improving the bounds of ifTTl [T2l . 



1.3 A framework for rounding via contention resolution schemes 

We now describe our framework for the problem maxggi f{S)- The framework assumes the following; (i) there 
is a polynomial-time value oracle for /, (ii) there is a solvable down-monotone polytope Pj that contains the set 
{I^IS" G I}, and (iii) there is a monotone c-balanced contention resolution scheme tt for Pj. Then we have the 
following simple algorithm; 

1. Using an approximation algorithm, obtain in polynomial time a point x* e Pi such that 

P(x*) > a ■ max{P(x) | x e Px n {0, 1}^} > a ■ iaa^f{S). 

2. Round the point x* using the CR scheme tt to obtain / = tTx* (P(x*)), and return its pruned version rif{I). 

Theorem 1.5. The preceding framework gives a randomized (a c) -approximation algorithm for maxsgj f{S), when- 
ever f is non-negative submodular, a is the approximation ratio for max{P(x) | x € Pj n {0, 1}^} and Px admits a 
monotone c-balanced CR scheme. If f is monotone then the pruning step is not needed. If f is modular then the ratio 
is c and the CR scheme is not even constrained to be monotone. 

Proof We have P(x*) > aOPT with OPT = maxsei f{S). TheoremOshows that E[f{T]f{I))] > cP(x*), hence 
E[/(f7/(^))] > acOPT. If / is monotone, the pruning step is not required by Theorem 1 1.3 1 

For modular /, P(x) is a linear function, and hence a = 1 can be obtained by linear programming. Moreover, if 
P(x) is a linear function, then by linearity of expectation, E[/(/)] > cP(x*) without any monotonicity assumption 
on the scheme. □ 

For non-monotone submodular functions. Theorem 1 1.1 1 gives a = 0.309; the currently best known approximation 
is (i — e) ~ 0.367 due to |[24 1 . For monotone submodular functions an optimal bound of a = 1 — is given in |48j. 

Combining schemes for different constraints: We are particularly interested in the case when I = nf^^Ii is the 

intersection of several different independence systems on A^; each system corresponds to a different set of constraints 
that we would like to impose. Assuming that we can apply the above framework to each Ii separately, we can obtain 
an algorithm forZ as follows. 

*We also describe the {b, 1 — 6) scheme in Section 1441 for completeness. This scheme is simpler and computationally advantageous when 
compared to the optimal scheme. 
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Lemma 1.6. Let I = C\^^ili and Px = n,Pxi- Suppose each Pj^ has a monotone (6, Ci)-balanced CR scheme. Then 
Px has a monotone (b, Yii Ci)-balanced CR scheme. In the special case that each element of N participates in at most 
k constraints and Ci = cfor all i then Px has a monotone (&, c^)-balanced CR scheme. Moreover, if the scheme for 
each Pxi is implementable in polynomial time then the combined scheme for Px can be implemented in polynomial 
time. 

Therefore, we can proceed as follows. Let Pi^ be a polytope that is the relaxation for 1;. In other words {I5 : 
5 e 2^} is contained in Px^. Let Px ~ C^iPxi- It follows that {I5 : 5 G 1} is contained in Pj and also that there 
is a polynomial-time separation oracle for Px if there is one for each Px-. Now suppose there is a monotone (6, q)- 
balanced contention resolution scheme for Px, for some common choice of h. It follows from Lemma IL6I that Px 
has a monotone (5, J^^ Ci)-balanced contention resolution scheme, which can be transformed into a {b J| ■ Cj)-balanced 
scheme for Px- We can then apply Theorem lL5l to obtain a randomized [ab c;) -approximation for max^gx f{S) 
where a depends on whether / is modular, monotone submodular or non-monotone submodular 

In this paper we focus on the framework with a small list of high-level applications. We have not attempted to 
optimize for the best possible approximation for special cases. We add two remarks that are useful in augmenting the 
framework. 

Remark 1.7. Whenever the rounding step of our framework is performed by a CR scheme that was obtained from a 
(6, c)-balanced CR scheme — in particular in the context mentioned above when combining CR schemes for different 
constraints — we can often strengthen the procedure as follows. Instead of approximately solving maxxep^ F[ii), 
we can approximately solve maxy^bp^ P{y) fo obtain y*, and then directly apply the (b, c)-balanced scheme to y*, 
without transforming it first to a bc-balanced scheme. This may be advantageous if the problem maxygbp^- P(y) 
admits a direct approximation better than one obtained by scaling from maxygp^- F{y). A useful fact here is that the 
continuous greedy algorithm for monotone submodular functions l\49\ ^ finds for every b € [0, 1] a point y* S bPx 
such that P(y*) > (1 - e"^) maXxGPx ^(^)- T^'^ indeed a stronger guarantee than the one obtained by first 
applying the continuous greedy to Px to obtain x*, and then used the scaled-down version bx*, which leads to a 
guarantee of only P(6x*) > &P(x*) > 6(1 — e~^) maxxep^ P(x). 

Remark 1.8. A non-negative submodular set function f is also subadditive, that is, f{A) + f{B) > f{A U B). In 
some settings when considering the problem maxggx f{S), it may be advantageous to partition the given ground set 
N into Ni, . . . , Nh, separately solve the problem on each Ni, and then return the best of these solutions. This loses a 
factor ofh in the approximation but one may be able to obtain a good CR scheme for each Ni separately while it may 
not be straightforward to obtain one for the entire set N. 

An application of the technique mentioned in Remark IL8I can be found in Section 14.81 where we use it in the 
context of column-restricted packing constraints. 

Organization: The rest of the paper is divided into three parts. Some illustrative applications of our framework 
are discussed in Section |2] Constant factor approximation algorithms for maximizing F over a solvable polytope 
are described in Section |3] Section |4] discusses the construction of CR schemes. This include a discussion of the 
connection between contention resolution schemes and correlation gap and its use in deriving optimal schemes for 
matroids. Furthermore, in the same section, we present CR schemes for knapsack constraints, sparse packing systems, 
and UFP in paths and trees. 

2 Applications 

In this section we briefly outline some concrete results that can be obtained via our framework. The meta-problem we 
are interested in solving is maxggx f{S) where Z is a downward-closed family over the given ground set N and / is 
a non-negative submodular set function over N . Many interesting problems can be cast as special cases depending on 
the choice of N, I and /. In order to apply the framework and obtain a polynomial-time approximation algorithm, we 
need a solvable relaxation Px and a corresponding (6, c)-balanced CR scheme. Note that the framework is essentially 
indifferent to / as long as we have a polynomial-time value oracle for it. We therefore focus on some broad classes of 
constraints and corresponding natural polyhedral relaxations, and discuss CR schemes that can be obtained for them. 
These schemes are formally described in Section |4] 
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Matroids and matchoids: Let M = [N, T) be a matroid constraint on N . A natural candidate for Pi is the integral 
matroid polytope {x € [0,1]" | x{S) < r{S),S C N} where r : 2^ Z+ is the rank function of M. We 
develop an optimal (1 — l/e)-balanced CR scheme for the matroid polytope. More generally, for any b G (0, 1] 
we design a (&, ^—^ — )-balanced CR scheme, which lends itself well to combinations with other constraints. The 
CR scheme for the matroid polytope extends via Lemma [TT6] to the case when I is induced by the intersection of 
k matroid constraints on N. A more general result is obtained by considering fc-uniform matchoids, a common 
generalization of fc-set packing and intersection of k matroids 1371 . defined as follows. Let G = (V, N) be a fc- 
uniform hypergraph; we associate the edges of the hypergraph with our ground set A^. For each v £ V, there is a 
matroid Aiy = {Nv,Iv) over A'^„, set of hyperedges in N that contain v. This induces an independence family I on 
where I ~ {S ^ N \ S f^ € Iy,v £ V}. fc-uniform matchoids generalize the intersection of k matroids in that 
they allow many matroids in the intersection as long as a given element of the ground set participates in at most fc of 
them. A natural solvable relaxation for I is the intersection of the matroid polytopes at each v. Via the CR scheme for 
the single matroid and Lemma [L6l we obtain a (6, {^—^ — )'^)-balanced CR scheme for any h e (0, 1] for fc-uniform 
matchoids. The choice of 6 = gives a e(fc+i) -balanced CR scheme for every fc-uniform matchoid. 

Knapsack / linear packing constraints: Let A^ = {1, 2, . . . , n}. Given a non-negative m x n matrix A and non- 
negative vector b, let X = {S \ Als < b} where I5 is the indicator vector of set S* C A^. It is easy to see that 
I is an independence family. A natural LP relaxation for the problem is Pj = {x | Ax < h,x ^ [0, 1]"}. The 
width of the system of inequalities is defined as W — [min^ j bi/Aij\. Some special cases of interest are (i) A is 
a {0, l}-matrix, (ii) A is column-restricted, that is, all non-zero entries in each column are the same and (iii) A is 
fc-column sparse, that is at most fc non-zero entries in each column. Several combinatorial problems can be captured 
by these, such as matchings and independent sets in graphs and hypergraphs, knapsack and its variants, and maximum 
throughput routing problems. However, the maximum independent set problem in graphs, which is a special case as 
mentioned, does not allow a n^^'^-approximation for any fixed e > 0, unless P=NP 1 301 . Therefore attention has 
focused on restricting A in various ways and obtaining upper bounds on the integrality gap of the relaxation Px when 
the objective function is linear. Several of these results are based on randomized rounding of a fractional solution and 
one can interpret the rounding algorithms as CR schemes. We consider a few such results below. 

• For a constant number of knapsack constraints (m = 0(1)), by guessing and enumeration tricks, one can 
"effectively" get a (1 — e, 1 — £)-balanced CR scheme for any fixed e > 0. 

• When A is fc-column sparse, there is a (6, 1 — 2fc6)-balanced CR scheme. If A has in addition width W > 2, 
there is a (&, 1 — fc(2e6)^~^) CR scheme for any b G (0, 1). These results follow from [R]. 

• When A is a {0, l}-matrix induced by the problem of routing unit-demand paths in a capacitated path or tree, 
there is a (6, 1 — 0{b)) CR scheme implicit in miHim. This can be extended to the unsplittable flow problem 
(UFP) in capacitated paths and trees via grouping and scahng techniques 11311 [141 1131 . 

Section|4]has formal details of the claimed CR schemes. There are other rounding schemes in the literature for 
packing problems, typically developed for linear functions, that can be reinterpreted as CR schemes. Our framework 
can then be used to obtain algorithms for non-negative submodular set functions. See fW\ for a recent and illuminating 
example. 

Approximation algoritlims. The CR schemes mentioned above when instantiated with suitable parameters and 
plugged into our general framework yield several new randomized polynomial-time approximation algorithms for 
problems of the form maxsei f{S), where / is non-negative submodular. We remark that these results are for some- 
what abstract problems and one can obtain more concrete results by specializing them and improving the constants. 
We have not attempted to do so in this paper. 

• If I is the intersection of a fixed number of knapsack constraints, we achieve a 0.309-approximation, improving 
the (0.2 — e) -approximation from ll35l and a recent (0.25 — £)-approximation ll34l . This is obtained via the 
(1 — e, 1 — £)-balanced CR scheme for a fixed number of knapsack constraints. 

• If Z is the intersection of a fc-uniform matchoid and £ knapsack constraints with £ a fixed constant, we obtain an 
f2(-|)-approximation (constant independent of £), which improves the bound of ^{j^) from 1281 . We remark 
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that this is a new result even for Unear objective functions. We obtain this by choosing b ~ il{l/k) and using 
the (6, {^—^ — )'"')-balanced CR scheme for fc-uniform matchoids and the (1 — e, 1 — £)-balanced CR scheme 
for a fixed number of knapsack constraints (this requires a separate preprocessing step). 

• If I is the intersection of a fc-uniform matchoid and an £-sparse knapsack constraint system of width W, we give 
an n{ fc^/i/w )-approximation, improving the f^(-p) approximation from ll28l . This follows by combining the 
CR schemes for fc-uniform matchoid and ^-column sparse packing constraints with a choice of & = n{ )■ 

• We obtain a constant factor approximation for maximizing a non-negative submodular function of routed re- 
quests in a capacitated path or tree. Previously an 0(1) approximation was known for linear functions ll5ll9][T4l 

3 Solving the multilinear relaxation for non-negative submodular functions 

In this section, we address the question of solving the problem max{i^(x) : x e P} where F is the multilinear 
extension of a submodular function. As we akeady mentioned, due to Bsl lTl. there is a (1 — 1 /e)-approximation for 
the problem max{F(x) : x e P} whenever F is the multilinear extension of a monotone submodular function and 
P is any solvable polytope. Here, we consider the maximization of a possibly non-monotone submodular function 
over a down-monotone solvable polytope. We assume in the following that P C [0, 1]^ is a down-monotone solvable 
polytope and F : [0, 1]^ R-|_ is the multilinear extension of a submodular function. We present two algorithms 
for this problem. As we noted in the introduction, there is no constant-factor approximation for maximizing non- 
monotone submodular functions over general — i.e., not necessarily down-monotone — solvable polytopes ||49l . The 
approximation that can be achieved for matroid base polytopes is proportional to 1 — where v is the fractional 
packing number of bases (see 1491 ), and in fact this trade-off generalizes to arbitrary solvable polytopes; we discuss 
this in Section |33] 

3.1 Continuous local-search 

Here we present our first algorithm for the problem max{i^(x) : x G P}. We remark that in the special case of 
multiple knapsack constraints, this algorithm is equivalent to the algorithm of ll35l . 

First we consider a natural local-search algorithm that tries to find a local optimum for F in the polytope P. For a 
continuous function g defined over a convex set C C M", a point x e C is a local optimum (in particular, a maximum), 
if _g(x) > 5(x') for all x' e C in a neighborhood of x. If g is differentiable over C, a first-order necessary condition 
for X to be a local maximum is that (y — x) • V(7(x) < for all y e C. If 5 is in addition a concave function then 
this is in fact sufficient for x to be a global maximum. However, in general the first-order necessary condition is not 
sufficient to guarantee even a local optimum. Although sufficient conditions based on second-order partial derivatives 
exist, it is non-trivial to find a local optimum or to certify that a given point x is a local optimum. Our algorithms 
and analysis rely only on finding a point which satisfies (approximately) the first-order necessary condition. Hence, 
this point is not necessarily a local optimum in the classical sense. Nevertheless, for notational convenience we refer 
to any such point as a local optimum (sometimes such a point is referred to as a constrained critical point). A simple 
high-level procedure to find such a local optimum for F(x) in P — which does not consider implementability — is the 
following. We will subsequently discuss how to obtain an efficient version of this high-level approach that returns an 
approximate local optimum. 

Algorithm 3.1. Continuous local search: Initialize x := 0. As long as there is y Cz P such that (y — x) • V-F(x) > 0, 
move X continuously in the direction y — x. If there is no such y Cz P, return x. 

This algorithm is similar to gradient descent (or rather ascent), and without considering precision and convergence 
issues, it would be equivalent to it. The importance of the particular formulation that we stated here will become more 
clear when we discretize the algorithm, in order to argue that it terminates in polynomial time and achieves a solution 
with suitable properties. 

The objective function F is not concave; however, submodularity implies that along any non-negative direction F 
is concave (see ll48l l7l). This leads to the following basic lemma and its corollary about local optima that we rely on 
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in the analysis of our algorithms. In the following x V y denotes the vector obtained by taking the coordinate-wise 
maximum of the vectors x and y; and x A y denotes the vector obtained by taking the coordinate-wise minimum. 

Lemma 3.2. For any two points x, y G [0, 1]^ and the multilinear extension F : [0, 1]^ — > M o/a submodular 
function, 

(y - x) • VF(x) > F(x V y) + F(x Ay) - 2i^(x). 

Proof. By submodularity, F is concave along any line with a nonnegative direction vector, such as (x V y) — x > 0. 
Therefore, 

F(x V y) - F{x) < ((x V y) - x) • VF(x), and similarly 
F(x Ay) - ^^(x) < ((x A y) - x) • VF(x), 

because of the concavity of F along direction (x A y) — x < 0. Adding up these two inequalities, we get F{x V y) + 
F(x Ay) - 2F(x) < ((x V y) + (x A y) - 2x) • V-F(x). It remains to observe that (x V y) + (x A y) = x + y, 
which proves the lemma. □ 

Corollary 3.3. Ifx is a local optimum in P, i.e. (y— x)-VF(x) < Oforally e P, then2F{x) > F(xVy) + i^(xAy) 
for any y £ P. 

3.2 Discretized local search 

What follows is a discretization of Algorithm 13.11 which is the one we actually use in our framework. Let M = 
max{f{i),f{N — i) : i £ N}. Notice that M is an upper bound on the maximum absolute marginal value of 
any element, i.e., M > msLXs^i |/s(«)l = inax{/(i) - f{$),f{N - i) - f{N) : i e N}. By subadditivity, we 
have 1/(5)1 < Mn for all S. It can be also verified easily that \§§-\ < M and < 2M for all i,j (see 

P9l ). We pick a parameter q = n"- for some sufficiently large constant a > 3 and maintain a convex combination 
X = i X]i=i where are certain points in P (without loss of generality vertices, with possible repetition). Each 
discrete step corresponds to replacing a vector in the convex combination by another Instead of the gradient VF(x), 
we use an estimate of its coordinates by random sampling. We use the following lemma to control the errors in 
our estimates. 

Lemma 3.4. Let F{x) = jj X^hLi fi^h) where Rh is a random set sampled independently with probabilities Xi. Let 
H = n2°+i, 5 = M/ji"-^ and M = max{/(i), /(TV - i) : i e N}. Then the probability that \F{x) - F{x)\ > 6 is 
at most 2e~"/^. 

Proof. Let us define Xh ~ 21771 (/(^'O ^ ^(^))' ^ random variable bounded by 1 in absolute value. By definition, 
E[Xh] =0. BytheChernoffbound,Pr[|X]Li^/i| > t] < 2e-*'/2H (gge Theorem A.1.16in 0]). We set iJ = n^^+i 
and* = in°+i, and obtain Pr[|F(x) - F{x)\ > M/n''-^] = Pr[| ^/^l > 5"""^^] < 26""/^ □ 

Given estimates of F(x), we can also estimate ^ = F(x V e,) - F((x V e,;) - ej) = E[/(i? + i) - f{R - i)]. 
The above implies the following bound. 

Corollary 3.5. Let 6 = M/n°'~^. If the total number of evaluations of F and is bounded by n'' and each estimate 

is computed independently using m?"-^^ samples, then with probability at least 1 — 0(n''e^"/^) all the estimates are 
within ±5 additive error . 

The algorithm works as follows. The input to the algorithm is a submodular function / given by a value oracle, 
and a poly tope P given by a separation oracle. 

Algorithm 3.6. Fractional local search. Let q = n°-,S = AI/n°-~^. Let x := | and initialize Vi = 

for all i. Use estimates V Fix) of V-F(x) within ±5 in each coordinate. As long as there w y £ P such that 
(y — x) • V-F(x) > A5n (which can be found by linear programming), we modify x := i X]?=i ^3' replacing one of 
the vectors in the linear combination by y, so that we maximize Fix). If there is no such y £ P, return x. 

Lemma 3.7. A Igorithm \3.6\ terminates in polynomial time with high probability. 
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Proof. We show that if all estimates of VF computed during the algorithm are within ±(5 in each coordinate — which 
happens with high probability — then the algorithm terminates in polynomial time. This implies the lemma since with 
high probability, we have that a polynomial number of estimates of VF are indeed all within ±5 in each coordinate. 
Hence, we assume in the following that all estimates VF of VF are within ±5. 

In each step, the algorithm continues only if it finds y G P such that (y — x) • VF(x) > ASri. Since WF 
approximates VF within ±5 in each coordinate, this means that (y — x) • VF(x) > 3Sn. Denote by x' a random 
vector that is obtained by replacing a random vector by y, in the linear combination x = ^ J^l^i ^i- The expected 
effect of this change is 

E[F(x') - F(x)] = ^ E (i^ (x + i(y - vO) - F(x)) = 7 - • ^^(^^) 

where x; is some point on the line between x and x+ i (y — v^), following from the mean-value theorem. Since q ~ n'^ 
and the second partial derivatives of F are bounded by 2M, we get by standard bounds that | |VF(xi) — VF(x)||i < 
2^ . 2M = ^ = 25n. Using also the fact that y - e [0, 1]", 



1 9 1 1 

E[F(x') - F(x)] > ^ V ((y - V,) • VF(x) - 25n) - ((y - x) • VF(x) - 25n) > - ■ Sn 

^-^ n n 



1=1 



using the fact that (y — x) • VF(x) > 3(5n. Therefore, if we exchange y for the vertex that maximizes our gain, we 
gain at least F(x') - F(x) > iJn = Also we have the trivial bound max F(x) < nM\ therefore the number 

of steps is bounded by n^"-^^. □ 



Lemma 3.8. Ifx is the output ofAlsorithm \3. 6\ then with high probability 

2F(x) > F(x V y) + F(x Ay) - bSn 

for every y G P. 

Proof. If the algorithm terminates, it means that for every y G P, (y — x) • VP(x) < ASn. Considering the accuracy 
of our estimate of the gradient VP(x) (with high probability), this means that (y — x) • VP(x) < 5(5n. By Lemma [l!2l 
we have (y - x) • VP(x) > P(x V y) + P(x Ay) - 2P(x). This proves the lemma. □ 



3.3 Repeated local search: a 0.25-approximation 

Next, we show that how to design a 0.25-approximation to the multilinear optimization problem using two runs of the 
fractional local-search algorithm. The following is our algorithm. 

Algorithm 3.9. Let x be the output of Algorithm \3 .6\ on the polytope P. Define Q = {yGP:y<l — x} and let z 
be the output of Alsorithm \3.6\ on the polytope Q. Return the better of Fix.) and F(z,). 

We use the following property of the multilinear extension of a submodular function. Let us replace each coordinate 
by a [0, 1] interval and let us represent a certain value Xi of the z'th coordinate by a subset of [0, 1] of the corresponding 
measure. 

Deflnition 3.10. Let X G , where C denotes the set of all measurable subsets o/[0, 1]. We say that X represents a 
vector X G [0, 1]^, if Xi has measure xi for each i £ N. 

From a "discrete point of view", we can imagine that each coordinate is replaced by some large number of elements 
M and a value of Xi is represented by any subset of size Mxi. This can be carried out if all the vectors we work with 
are rational. In the following, we consider functions on subsets of this new ground set. We show a natural property, 
namely that a function derived from the multilinear extension of a submodular function is again submodular. (An 
analogous property in the discrete case was proved in Il39]l35]| .) 
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Lemma 3.11. Let F : [0, 1]^ M.be a multilinear extension of a submodular function f. Define a function F* on 
, by F*{X) = F{'x), where x € [0, 1]^ is the vector represented by X. Then F* is submodular: 

F*{x l^y) + F*{Xf^y)< f*{x) + F*{y), 

where the union and intersection is interpreted component-wise. 

Proof. We have F(x) = E[/(x)] where Xi = 1 independently with probability Xi. An equivalent way to generate x is 
to choose any set X G representing x, generate uniformly and independently a number Vi e [0,1] for each i € N, 
and set Xi — 1 iff ri G Xi. Since the measure of Xi is Xi, Xi — 1 with probability exactly Xi. Therefore, 

F*{X) = F(x) = E[/(x)] = E[/({^ : r, € A",})]. 

Similarly, 

F*{y) = nf{{^--neyi\)]. 

This also holds for A" U J and X r^y■. since {X U y)i = X.,\J yi and {X n y)^ = A", n 3^;, we get 

F*{X U 3^) = E[/({i : r, e A",} U {z : r, e 3^,})] 

and 

F*(A' n 3^) - E[/({z : n e X,} n{i:ne y^})]. 

Hence, by the submodularity of /, 

F*ix uy) + F*{xny) = e[/({z : n e x,} u{i:ne y,}) + /({i : n e x,} n {i n e yi})] 

< E[f({^■.neX,}) + f{{^:ney^})] 

= F*{X)+F*{y). 

□ 

From here, we obtain our main lemma - the average of the two fractional local optima is at least -jOPT. 

Lemma 3.12. Let OPT = max{_F(x) : x G P}. Let x be the output of Alsorithm \3.6\ on polytope P, and z an output 
of Algorithm 13.61 on polytope Q = {yEP:y<l^ x}, with parameter S as in Alsorithm \3.6\ Then with high 
probability 2F(x) + 2i^(z) > OPT - lQ5n. 

Proof. Let OPT = F(x*) where x* G P. By Lemma lsTSl the output of the algorithm x satisfies with high probability 

2i^(x) > F(x Vx*) +i^(x Ax*) - 5(5n. (1) 

In the restricted polytope (5 = {yGP:y<l — x}, consider the point z* = (x* — x) V G Q. Again by Lemma [3781 
the output of the algorithm z satisfies 

2F(z) > i^(zVz*) +F(z Az*) - 5^n. (2) 

Now we use a representation of vectors by subsets as described in Def. 13.101 We choose X, X* ,Z, Z* G to 
represent x, x*, z, z* as follows: for each i £ N, Xi = [0, Xi), Zi = [xi,Xi + Zi) (note that Xi + Zi < 1), X* = [0, x*) 
and Z* = [0, z*) = [0, max{a;* - x„0}). Note that {X n Z), = for all i G N. 

Defining F* as in Lemma[3TI] we have F*{X) = i^(x), F*{X*) = F(x*) = OPT, F*{Z) = F(z) and 
F*{Z*) = F{z*). Using relations like [0,x,)U [0,x*) = [0, max{a;,, x*}), we also get F*(A' U A"*) = F(xVx*) 
andF*(A'nA'*) = i^(xAx*). Furthermore, we have {X*\Xi)UZi = [xi, max{x* , Xi+Zi}) = [xi, Xi+uia-x{z* , Zi}). 
This is an interval of length max{<,z,} = (z V z*),; and hence i^*((A'* \ A") U Z) = i^(z V z*), where (A"* \ A") U Z 
is interpreted component-wise. 

The property of the first local optimum ([T) can be thus written as 2F(x) > F*{X U X*) + F*{X n X*) ~ 55n. 
The property of the complementary local optimum (|2) can be written as 2F(z) > F*{{X* \ A") U Z) — 56n (we 
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discarded the nonnegative term F(z A z* ) which is not used in the following). Therefore, 2F(x) + 2F(z) > F*{X U 
X*) + F*{Xr\ X*) + F*{{X* \X)\JZ)- IQSn. By Lemma[3lll F* is submodular Hence we get 

F*{X n X*) + F*{{X* \ A") U Z) > F*{{X n X*) U {X* \X)\JZ) 

= F*{X*UZ) 

(we discarded the intersection term). Finally, using the fact that A" n Z = and again the submodularity of F*, we get 

F*{X U X*) + F*{X* U Z) > F*{{X U X*) n {X* U Z)) = 
(we discarded the union term). To summarize, 

2F{x) + 2F{z) > F*{X U X*) + F*{X n X*) + F*{{X* \X)UZ)- lOSn 

> F*{X U X*) + F*{X* U Z) - lOSn 

> F*{X*) - lOSn = OPT - lOSn. 

□ 

Since the parameter S in Algorithm 13.61 is chosen as 5 = -^—^ for some constant a > 3, and the optimum is 
trivially bounded by OPT < Mn, we obtain the following. 

Corollary 3.13. For any down-monotone polytope P C [0, 1]^ and multilinear extension of a submodular function 
F : [0, 1]^ — > R+, A Isorithm \3.9\ sives with high probability a (i — o(l))-approximation to the problem max{_F(x) : 
xe P}. 

3.4 Restricted local search: a 0.309-approximation 

Next, we present a modified local-search algorithm which is a generalization of the algorithm for matroid polytopes 
from f49\. We remark that this algorithm is in fact simpler than the ^-approximation from the previous section, in 
that it does not require a second-stage complementary local search. Both algorithms work for any down-monotone 
polytope P. However, our analysis of the restricted local-search algorithm is with respect to the best integer solution in 
the polytope; we do not know whether the approximation guarantee holds with respect to the best fractional solution. 

Algorithm 3.14. Fix a parameter t G [0,1]. Using Algorithm\3^ find an approximate local optimum x. in the polytope 
Pn [0,i]^. Return X. 

We show that with the choice of t = ^(3 — Vb), this algorithm achieves a i(— 1 + a/5) — 0.309-approximation 
with respect to the optimal integer solution in P. 

Lemma 3.15. Let x be an output of Algorithm \3.6\ on P n [0, t]^ . Define w g [0, 1]^ by Wi = t if Xi > t — 1/n and 
Wi = 1 if Xi < t — 1/n. Let z be any point in P and let z' = z A w. Then with high probability, 

2P(x) > P(x V z') + P(x A z') - bSn^. 

We remark that the above inequality would be immediate from Lemma [3781 if z' e P n [0, t]^. However, z' is not 
necessarily constrained by [0, t]^ . 

Proof. Consider z' = z A w as defined above. By down-monotonicity, z' e P. Also, the coordinates where z[ > t are 
exactly those where < t — So we have x + i(z' — x) G P n [0, t]^ . By the stopping rule of Algorithm 13. 61 

-(z'-x) • VP(x) < 5(5n. 
n 

By Lemma[l2l this implies P(x V z') + P(x A z') - 2P(x) < (z' - x) • VP(x) < hSn'^. □ 
In the rest of the analysis, we follow P9l . 
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Definition 3.16. For x G [0, 1]^ and A G [0, 1], we define the associated "threshold set" as T>a(x) = {i : Xi > A}. 
Lemma 3.17. Let x e [0, 1]^. For any partition N = CUC, 

F(x) > e[/((t>a(x) n c) u (t>v(x) n c))] 

where A, A' G [0, 1] are independently and uniformly random. 

This appears as Lemma A. 5 in |49 |. We remark that the right-hand side with C = % o\: C = N gives the Lovasz 
extension of / and the lemma follows by comparing the multilinear and Lovasz extension. For a non-trivial partition 
(C, (7), the lemma follows by two applications of this fact. The next lemma is exactly as in P9l for the special case of 
a matroid polytope; we rephrase the proof here in our more general setting. 

Lemma 3.18. Assume that t G [0, ^(3 — a/S)]. Let x be an output of Alsorithm \3.6\ on P H [0, t]^ (with parameter 
a > 4), and let z ~ Ic be any integer solution in P. Then with high probability, 

nx)> (i-it^-o(l)) f{C). 

Proof. Define A = {i : xi > t— 1/n} and let w = tl^ + 1^, z' = z A w as in Lemma [3.15l Since z = l^, we have 
z' = tlAnC + lc\A- By Lemma lTTSl we get 

2i^(x) > i^(xVz') +F(xAz') - 5(5n^ (3) 

First, let us analyze F{x A z'). Since z' = tlyinc + lc\A and x G [0, t]^ , we have x A z' = x A Ic- We apply 
Lemma 13.171 which states that 

^^(xAz') = F(xAlc) >E[/(r>A(x)nC7)]. 

Due to the definition of Ty\{x), with probability t — 1/n we have X < t — 1/n and T>>,(x) contains A ^ {i : Xi > 
t - 1/n}. Then, /(r>A(x) n C) + /(C \ A) > f{C) by submodularity. We conclude that 

FixAz')>(t--){f{C)-f{C\A)). (4) 



n ^ 

Next, let us analyze F(x V z'). We apply Lemma l3.17l We get 

F(x V z') > E[/((r>A(x V z') n C) U {Tyy (x V z') n C))]. 

The random threshold sets are as follows: Ty\{x V z') n C = Ty\{z') is equal to C with probability t, and equal 
to C \ A with probabiHty 1 - <, by the definition of z'. r>A' (x V z') n C* = r>A' (x) n C is empty with probabiUty 
1 — t, because x G [0, t]^ . (We ignore the contribution when r>A'(x) n C* 7^ 0.) Because A, A' are independently 
sampled, we get 

^^(x V z') > (1 - t){tf{C) + (1 - t)f{C \ A)). 
Provided that t G [0, ^(3 — \/5)], we have t < (1 — t)^. Then, we can write 

F{xyz')>t{l-t)f{C)+tf{C\A). (5) 

Combining equations (O, (|4]i and (|5j, we get 

2F(x) > F(xVz') + f(xAz') -5(5n2 

> t(l - t)fiC) + tfiC\A) + {t- -)ifiC) - fiC \ A)) - 5Sn^ 

n 

> {2t^t^^o{l))f{C) 

using Sn^ = M/n"-^ = o(l)/(C) for a > 4. □ 

CoroUary 3.19. Fort = ^(3— \/5'), Alsorithm \3.14\ with hish probability achieves a j{—l+V5—o{l))-approximation 
with respect to max{F(x) : x G P H {0, 1}^}. 
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3.5 Approximation for general polytopes 

In this section, we formulate an approximation result for the problem max{F(x) : x e P} when P is a general 
solvable polytope (not necessarily down-monotone). This result is included only for the sake of compleness; we do 
not have any concrete applications for it. Our result generalizes (while losing a factor of 4) the result for matroid base 
polytopes from 1491 . which states that a ^ (1 — ^ — o(l))-approximation can be achieved, provided that the fractional 
base packing number is at least i/ where G [1, 2]. As observed in P9l , the fractional base packing number being at 
least 1/ is equivalent to the condition P Ci [0, ^]'^ 0. This is the condition we use for general polytopes. We state the 
algorithm only in its continuous form; we omit the discretization details. 

Algorithm 3.20. Let t e [0, 1] be a parameter such that P n [0, i]^ ^ 0. Initialize x S P n [0, t]^ arbitrarily. As 
long as there is y € PR [0,^(1 + i)]^ such that (y — x) • VP(x) > (which can be found by linear programming), 
move X continuously in the direction y — x. If there is no such y G P H [0, ^{1 + t)]^ , return x. 

Note that even though we require Pn [0, t]^ ^0, the local search works inside a larger polytope Pn [0, i (1 + 
This is necessary for the analysis. 

Theorem 3.21. For any solvable polytope such that P H [0,t]^ 7^ 0, Algorithm 15.201 approximates the problem 
max{P(x) : x £ P} within a factor of ^{1 — t). 

Proof. The algorithm maintains the invariant x G P D [0, ^(1 + i)]^. Suppose that the algorithm returns a point x. 
Then we know that for every y G P n [0, ^(1 + t)]^ , (y — x) • VP(x) < 0. We use a particular point y defined 
as follows: Let x* be the optimum, i.e. P(x*) = max{P(x) : x e P}, and let Xq be any point in P n [0, t]^ , for 
example the starting point. Then we define y = i(xo + x*). By convexity, we have y G P,andsincex* e [0,1]^, we 
also have y G [0,^(1 + 1)]^ . Therefore, by the local-search condition, we have (y — x) • VP(x) < 0. By Lemma [J!2l 

2P(x) > P(x V y) + P(x A y) > P(x V y). 

Let x' = X V y. The point x' has the following properties: x' = x V ^(xo + x*) > ^x*, and also x' e [0, 1)]^ . 

Consider the ray ^x* + ^(x' — ^x*) parameterized by ^ > 0. Observe that this ray has a positive direction in all 
coordinates, and it is possible to go beyond = 1 and still stay inside [0, 1]^: in particular, for ^ = we get a point 
ix* + ]-^(x' — 5X*) < ]-^x' G [0, 1]^. Using this fact, we can express x' as a convex combination: 

1 + * /I , 2 ^ , 1 , \ l-t 1 , 

-X* H (x' X* H -X* 

2 l + V 2 7 2 2 

(the reader can verify that this is an identity). By the concavity of F in positive directions, we get 

P(x') > 1±1f fix* + ^(x' - ix*)) + '-If fix' 
' - 2 \2 1 + r 2 7 2 V2 

As we argued, ix* + y^Cx' — ^x*) G [0, 1]^, so we can just lower-bound the respective value by 0, and we obtain 



P(x')>1^f(M >i^P(x*). 



Finally, our solution satisfies 



P(x) > ip(xVy) = Ip(x') > ^-^F{^*) = l^OPT. 



□ 



4 Contention resolution schemes 

In this section we discuss contention resolution schemes in more detail and prove our results on the existence of 
contention resolution schemes and their application to submodular maximization problems. 
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4.1 Contention resolution basics 



Recall the definition, from Section [T] of a {b, c)-balanced CR scheme tt for a polytope Pj. We first prove the claim 
that a {b, c)-balanced CR scheme tt can be transformed into a bc-balanced CR scheme tt' as follows. Let x E Px and 
A C TV. We define tt'-x^{A) as follows. First each element of A is removed independently of the others with probability 
1 — 6 to obtain a random set A' C A. We then set tt'^{A) = ttx{A'). The key observation is that if A is a set drawn 
according to the distribution induced by i?(x), then A' has a distribution given by R{bx). Hence, for any i G N 



Pr[i e <(i?(x)) I i G i?(x)] 



Pr[?: e <(i?(x))] _ Pr[i e Tr^{R{bx))] _ &Pr[i e TTjR{bx))] 

Pr[i e R{x)] ^ Pr[i E R{x)] ^ Pr[i e R{bx)] 
bPi-[i e TT^{R{bxj) I i e R{bx)] > be, 



where the last inequality follows from the fact that tt is (b, c)-balanced. 

Monotonicity of CR schemes for submodular function maximization: The inequality that relates contention resolu- 
tion to submodular maximization is given in Theorem ll.3l A proof of this inequality also appears in ID for monotone 
functions without the pruning procedure. Before presenting the proof, we provide some intuition on why monotonicity 
of the CR scheme is needed in the context of submodular function maximization, and we specify the pruning proce- 
dure 11 f. It is easy to see that if Px has a c-balanced CR scheme then it implies a c-approximation for maximizing 
a linear function over Px- If x is a fractional solution then its value is WiXj, where Wi are some (non-negative) 
weights; since each element i is present in the final solution produced by a c-balanced CR scheme with probability 
at least cXi, by linearity of expectation, the expected weight of a solution returned by a c-balanced scheme is at least 
WiXi. More generally, we would like to prove such a bound for any submodular function / via F. However, 
this is no longer obvious since elements do not appear independently in the rounding scheme; recall that F{x) is the 
expected value of / on a set produced by independently including each i with probability Xi. Monotonicity is the 
property that is useful in this context, because elements of smaller sets contribute more to a submodular function than 
elements of larger sets. 

To prove Theorem 11.31 we first introduce the claimed pruning function rjf. To prune a set / via the pruning 
function rjf, an arbitrary ordering of the elements of N is fixed: for notational simplicity let = {1, . . . ,n} which 
gives a natural ordering. Starting with J = the final set J = 77/(/) — which we called the pruned version of / — is 
constructed by going through all elements of / in the order induced by TV. When considering an element i, J is 
replaced by ,7 + i if /(J + i) - f{J) > 0. 

Proof of Theorem \l.3\ Let R = i?(x) and / = tTx{R), and let J = ?//(7) if / is non-monotone and J = I otherwise. 
Hence, in both cases, J is the set returned by the suggested rounding procedure. The proof we present works for both 
cases, but we will mainly concentrate on the more compUcated non-monotone case, where the pruning is needed, and 
explain why the different steps also hold for the monotone case without pruning. 

Assume that TV = {1, . . . , n} is the same ordering of the elements as used in the pruning operation (in case no 
pruning was appUed, any order is fine). Notice that if J is pruned, we have for i G I, 

/jn[j-i] (i) > if and only if i e J. (6) 
The main step that we will prove is that for any fixed i G {1, . . . ,n}, 

E[/( J n [z]) - f{J n [z - 1])] > cE[/(i? n W) - f{R n[i- i])]. (7) 

The theorem then follows from ([7) since 

n n 

E[/( J)] = /(0) + E n [z]) - /(J n [i - 1])] > /(0) + c E[/(i? n H) - f{R n [z - i])] > cE[/(i?)]. 

i=l i=l 
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Hence, it remains to prove dTji. Consider first the non-monotone case. Here we have 



E[/( J n [i]) - /(J n [z - 1])] = E[l,e j/jn[.-i] (*)] 

= Pr[z e i?] • E[l,e,7/jn[.,-i] W h e i?] 

i Pt[i e i?] • E[l,e.j max{0, /,7n[.-i] (*)} M e -R] 

i Pr[i e R] ■ E[l,ei max{0, /,7n[.-i](*)} M e ^] 

> Pr[i e i?] • E[lig7 max{0, /i?,n[i-i] (*)} | i G (since / is submodular) 

= Pt[i e i?] • E[E[l,e/ max{0, fRn[^-l]ii)} \ R] \ i € R] 
= Pr[z e i?] -ElElL^e/ | R]ma^{OJnr,[^-l]{i)} \ i € R]. 

However, the two equalities based on (|6]l hold as well in the monotone case without pruning, since /,/n[i-i](*) ^ 
due to monotonicity, and / = J. 

On the product space associated with the distribution of R conditioned on i E R, both of the terms E[lig/ | R] 
and max{0, /i?n[j-i] (*)} ^""e non-increasing functions, because of the monotonicity of the CR scheme used to obtain 
/ from R and / being submodular, respectively. Hence, by the FKG inequality we obtain 

Pt[i G R] ■ E[E[l,e, I R] max{0, fRn[^-l]i^)} \ i & R] 

> Pt[i e R]-nUei I i G R]■nma^{OJnnl^-l]i^)} \ ^ ^ R] 
= PT[t G R] ■ Pr[i eI\teR] • E[max{0, fRnl^-l]{i)} \ i e R] 

> cPiit G R] ■ E[max{0, /fln[.-i](0} 1* e i?] 
>cPT[teR]-E[fRni,~^{i) \teR] 

= cPT[ie R]■E[fnnl^-lM 

= cE[/(i?n W)-/(i?n[j-i])], 

where in the second to last equality we use again the property that /i7n[j-i](*) is independent of i G R. Hence, this 
shows dTji as desired, and completes the proof. □ 

An alternative way to round in the context of non-monotone submodular functions, that does not rely on pruning, 
can be obtained by using a stronger notion of CR schemes. More precisely, we say that a (6, c)-balanced CR scheme 
TT for Px is strict, if it satisfies the second condition of a CR scheme with equality, i.e., Pr[i G 7rx(i?(x))] = c. We 
have the following (the proof can be found in Appendix lAli. 

Theorem 4.1. Let f : 2^ — > M+ be a non-negative submodular function with multilinear relaxation F, and x. be a 
point in Px, a convex relaxation fori C 2^. Let n be a monotone and strict {b, c)-balanced CR scheme for Px, and 
let I = 7rx(i?(x)). Then 

E[/(/)]>cF(x). 

The advantage of using a strict CR scheme compared to applying the pruning step is that this version of rounding is 
oblivious to the underlying submodular function /. This could potentially be useful in settings where one is interested 
in simultaneously maximizing more than one submodular function. Assume for example that x is a point such that 
Fi (x) and F2 (x) have simultaneously high values, where Fi and F2 are the multilinear relaxations of two submodular 
functions /i and /2 . Then using a rounding that is oblivious to the underlying submodular function leads to a randomly 
rounded set / satisfying E[/i(/)] > ci^i(x) and E[/2(/)] > ci^2(x). 

Any monotone but not necessarily strict {b, c)-balanced CR scheme tt can be transformed into a monotone (6, c)- 
balanced CR scheme that is arbitrarily close to being strict as follows. For each element i £ N, one can estimate 
the probability c'j = Pr[i G 7rx(i?(x)) | « G /] > c via Monte-Carlo sampling within a polynomially small error 
(assuming that c is a constant). Then we can modify the CR scheme by removing from its output /, element i G I with 
probability 1 — c/c^. The resulting scheme is arbitrarily close to being strict and can be used in place of a strict scheme 
in Theorem 14. II with a weaker guarantee; in applications to approximation, the ratio is affected in the lower-order 
terms. We omit further details. 
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Combining CR schemes: Next, we discuss how to combine contention resolution schemes for different constraints. 
We consider a constraint X = nf^iXi and its polyhedral relaxation Px = f^iPii, such that Px^ has a monotone (6, q)- 
balanced CR scheme tt*. We produce a contention resolution scheme tt for I which works with respect to the natural 
combination of constraint relaxations — an intersection of the respective polytopes Pj. . This ensures that the relaxed 
problem is still tractable and we can apply our optimization framework. 

In case some elements D C N are not part of the constraint li, we assume without loss of generahty that tt* never 
removes elements in D, i.e., 7r^(A) O D = AO D for any x e bPx^ and A C N. The combined contention resolution 
scheme tt for Px is defined by 

TT^(A) = Pi for a C iV, X e bPx. 

i 

A straightforward union bound would state that the combined scheme tt is (&, 1 — ^ Ci))-balanced for Px- 

Using the FKG inequality, we obtain a stronger result in this setting, namely a (6, c j-balanced scheme. Moreover, 
if each constraint admits a (fo, c)-balanced scheme and each element participates in at most k constraints, then we 
obtain a (6, c'°)-balanced scheme. This is the statement of Lemma [L6l which we prove here using the combined 
scheme tt defined above. 

Proof of Lemma \L^ Let us consider the < k constraints that element i participates in. For simplicity we assume 
k = 2; the general statement follows by induction. For notational convenience we define R — i?(x), Ii — tt^{R) and 

Conditioned on R, the choices of Ii , I2 are independent, which means that 

Pr[i e /i n /2 I i?] = Pr[?: e /i & i e /z | i?] = Pr[?: eh\R] Pr[i ehlR]. 

Taking an expectation over R conditioned on i e R, we get 

Pr[t e /i n /2 U e i?] = Efi[Pr[t e hn h \ R] \ i e R] = Efi[Pr[i eh\R] Pr[i el2\R]\ieR]. 

Both Pr[i e /i I i?] and Pr[j G I2 \ R] are non-increasing functions of R on the product space of sets containing i, so 
by the FKG inequality, 

Ei^[Pr[^ eIi\R] Pr[i e h \ R] \ i e R] > Efi[Pr[i e h \ R] \ i e R] ■ ER[Pr[i e h \ R] \ i e R]. 
Since these expectations are simply probabilities conditioned oni E R, we conclude: 

PT[i e /i n /2 M e i?] > Pr[i eli\ieR] Pr[i e /a | i e i?]. 

Monotonicity of the above scheme is also easily implied: consider j G Ti C T2 C N, then 

Pr[j e I\R = Ti]=ll Pr[j e h\R = T,] > J] Pr[j e h\R = T2] = Pr[j G I\R = T^]. 

i i 

where the inequality follows from the fact that each of the schemes is monotone. The polynomial time implementability 
of the composed scheme follows easily from the polynomial time implementability of tt^ and tt^. □ 

4.2 Obtaining CR schemes via distributions of deterministic CR schemes 

We now describe a general way to obtain CR schemes relying on an LP approach. Recall the formal definition 
of CR schemes given in Definition 11.21 in particular the differences between oblivious, deterministic and general 
(randomized) schemes. First, we note that the simplest CR schemes are the oblivious ones. An oblivious scheme does 
not depend on x and is deterministic; hence it is essentially a single mapping ti : 2^ ^ I that given A (- N returns 
a set 'n{A) such that 7r(A) C A and 7r(A) € I. Several alteration based schemes are oblivious — see ||5] [14] for 
some examples. A typical oblivious scheme fixes an ordering of the elements of (that depends on the combinatorial 
properties of T)\ it starts with an empty set A' , and considers the elements of A according to the fixed order and 
adds the current element i to the set A' if A' U {i] S X, otherwise it discards i. Finally it outputs A' . These greedy 
ordering based insertion schemes are easily seen to be monotone. A deterministic CR scheme is more general than 
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an oblivious scheme in that the output can depend on x; in other words, for each x G Pj, tTx is a mapping from 
2^ to I. The advantage or need for such a dependence is demonstrated by matroid polytopes. Let P{A4) be the 
convex hull of the independent sets of a matroid Ai; oblivious schemes cannot give a c-balanced CR scheme for any 
constant c. However, we can show that for any b G [0, 1] a good deterministic CR scheme exists: for any x e Pm, 
there is an ordering CTx that can be efficiently computed from x such that a greedy insertion scheme based on the 
ordering CTx gives a (6, 1 — 6)-balanced scheme. Such a scheme for b ~ 1/2 is implicitly present in ifTTl . however for 
completeness, we give the details of our scheme in Section l4!4l The algorithm in ifTOl for geometric packing problems 
was reinterpreted as a deterministic CR scheme following our work; it is also based on computing an ordering that 
depends on x followed by a greedy insertion procedure via the computed ordering (see also more recent work [19]). 
Such ordering based deterministic schemes are easily seen to be monotone. 

In contrast to deterministic schemes, general (randomized) CR schemes are such that 7rx(A) is a random feasible 
subset of A. Randomization is necessary to obtain an optimal result even when considering contention for a single 
item II22II23I . For the time being, we do not require the CR schemes to be monotone; this is a point we discuss later. A 
non-oblivious (6, c)-balanced CR scheme tt, deterministic or randomized, can depend on x, and hence it is convenient 
to view it is a collection of separate schemes, one for each x G bPx- They are only tied together by the uniform 
guarantee c. In the following we will fix a particular x and focus on finding the best scheme tTx for it. As we already 
discussed, if tt is deterministic, then tTx is a mapping from 2^ to I. We observe that a randomized scheme tTx is a 
distribution over deterministic schemes; note that here we are ignoring computational issues as well as monotonicity. 
We formalize this now. Call a mapping (p from 2^ to I valid if (j){A) C A \/A C N. Let be the family of all 
valid mappings from 2^ to I. Any probability distribution (A^), induces a randomized scheme tTx as follows. 

For a set A, the algorithm tTx first picks according to the given probability distribution and then outputs 

Conversely, for every randomized scheme tTx, there is an associated probability distribution (A^), £ $* 0. Based on 
the preceding observation, one can write an LP to express the problem of finding a CR scheme that is {b, c)-balanced 
for X with a value of c as high as possible. More precisely, for each € $*, we define gi,^ = Pr[i € (f>{R)], where, as 
usual, R := i?(x) is obtained by including each j G N in R with probability Xj, independently of the other elements. 
Thus, for a given distribution (A^)^^**, the probability that the corresponding CR scheme ttx returns a set ttx{R) 
containing i, is given by J2,pe<s>* 1i-'t>^4>- Hence, the problem of finding the distribution {X^)-^^'!'-' that leads to a (6, c)- 
balanced CR scheme for x with c as high as possible can be formulated as the following linear program (LPl), with 
corresponding dual (DPI). 

XiC \/i G N 
1 

V0 e $* 

n V(/) e $* 
1 

yie N 

In general we may also be interested in a restricted set of mappings $ C $*. In the above LP we can replace $* 
by $ to obtain the best c that can be achieved by taking probability distributions over valid mappings in $. Let c(x, $) 
be the optimum value of the LP for a given x and a set $ C $*. It is easy to see that c(x, $) < c(x, $*) for any $. 
From the earlier discussion, c(x, $* ) is the best scheme for x. We summarize the discussion so far by the following. 

Proposition 4.2. There exists a (6, c)-balanced CR scheme for Pi iff infxebPx c(x, $*) > c. 

Proving the existence of a (6, c)-balanced CR scheme: To show that Pj has a (6, c)-balanced CR scheme we need 
to show that c(x, $*) > c for all x e bPx- By LP duality this is equivalent to showing that the optimum value of the 
dual (DPI) is at least c for all x. We first reformulate the dual in a convenient form so that proving a lower bound c on 

'Let k be an upper bound on the number of random bits used by ttx- For any fixed string r of k random bits, let </>'" be the valid mapping from 
2^ to I generated by the algorithm ttx with random bits set to r. The distribution where for each r the probability assigned to is 1/2'^ is the 
desired one. 



max c 



mm 

(DPI) 



Vi > 
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the dual optimum reduces to a more intuitive question. We will then address the issue of efficiently constructing a CR 
scheme that nearly matches the lower bound. 

Below we will use R to denote a random set obtained by picking each i G iV independently with probability x; 
and use probabilities and expectations with respect to this random process. The optimum value of the dual can be 
rewritten as: 



mm max = mm max — —= — mm max 



y>o 0e** J2teN ^iyi "^"^^ 
For any fixed weight vector y > we claim that 



max Eij; 



y>0 ^e** 



E 



J2 y^ 




ie4>{R) 





max 



which follows by considering the specific mapping G $* that for each A Q N sets (t){A) = max^'cA,A'gi y(^) 
Thus, the dual optimum value is 

. Ei^ [maxsci?.,SGi E^es yi] 



mm ■ 

y>0 



EieJV ^iyi 



(8) 



The above expression can be explained as an "integrality gap" of Pj for a specific rounding strategy; here the 
problem of interest is to find a maximum weight independent set in I. The vector y corresponds to weights on N . The 
vector X corresponds to a fractional solution in bPx (it is helpful here to think of 6 = 1). Thus Eigtv value 
of the fractional solution. The numerator is the expected value of a maximum weight independent set in R. Since we 
are minimizing over y, the ratio is the worst case gap between the value of an integral feasible solution (obtained via 
a specific rounding) and a fractional solution. 

Thus, to prove the existence of a (6, c)-balanced CR scheme it is sufficient (and necessary) to prove that for all 
y > and x e hPi 

Er(x) [maxscfl,SGi E^es 2^*] 



EieJV '^iyi 
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Constructing CR schemes via the ellipsoid algorithm: We now discuss how to efficiently compute the best CR 
scheme for a given x by solving (LPl) via the dual (DPI). We observe that c(x, $*), the best bound for a given 
x, could be smaller than the bound c. It should not be surprising that the separation oracle for the dual (DPI) is 
related to the preceding characterization. The separation oracle for (DPI) is the following; given i^i and weight vector 
y, normalized such that ^j^Xii/i ~ 1, check whether there is any such that "^i^j^ li^ipVi > M if so 

output a separating hyperplane. To see whether there is a violated constraint, it suffices to evaluate max^g$. g^.^?/; 
and compare it with /i. Following the previous discussion, this expression is equal to E/j.(x) [niaxscij.sei Eies y«] ■ 
One can accurately estimate this quantity as follows. First, we sample a random set R using marginals given by 
x. Then we find a maximum y-weight subset of R that is contained in I. This gives an unbiased estimator, and 
to get a high-accuracy estimate we repeat the process sufficiently many times and take the average value. Thus, the 
algorithmic problem needed for the separation oracle is the maximum weight independent set problem for I: given 
weights y on iV and a ^ C output a maximum weight subset of A in I. The sampling creates an additive error e in 
estimating E/j(x) [ina'X5Ci?,sei Eies yi] which results in a corresponding loss in finding the optimum solution value 
II* to (DPI). To implement the ellipsoid algorithm we also need to find a separating hyperplane if there is a violated 
constraint. A natural strategy would be to output the hyperplane corresponding to the violating constraint found 
while evaluating max^gg,* (li,4,Vi- However, we do not necessarily have the exact coefficients g,; ^ for the constraint 
since we use random sampling. We describe in Section |B] of the appendix the technical details in implementing 
the ellipsoid algorithm with sufficiently accurate estimates obtained from sampling. For now assume we can find a 
separating hyperplane corresponding to the most violated constraint. The ellipsoid algorithm can then be used to find 
a polynomial number of dual constraints that certify that the dual optimum is at least /i* — e where ji* is the actual 
dual optimum value. By strong duality /i* = c(x, $*). We then solve the primal (LPl) by restricting it to the variables 
that correspond to the dual constraints found by the ellipsoid algorithm. This gives a primal feasible solution of value 
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c(x, $*) — £ and this solution is the desired CR scheme. We observe that the primal can be solved efficiently since 
the number of variables and constraints is polynomial; here too we do not have the precise coefficients ^ but we 
can use the esimates that come from the dual — see Section |B] To summarize, an algorithm for finding a maximum 
weight independent set in I, together with sampling and the ellipsoid algorithm, can be used to efficiently find a 
(fe, c(x, $*) — e)-balanced CR scheme where e is an error tolerance; the running time depends polynomially on the 
input size and 1 /e. The proof can be easily adapted to show that an a-approximation for the max-weight independent 
set problem gives a a ■ c(x, $*) — e CR scheme. 

Monotonicity: The discussion so far did not consider the issue of monotonicity. One way to adapt the above approach 
to monotone schemes is to define $ to be the family of all deterministic monotone CR schemes and solve (LPl) re- 
stricted to $. A deterministic scheme is monotone if it has the property that i G (p{A) implies that i e ipiA') for all 
A' C A. Distributions of deterministic monotone schemes certainly yield a monotone CR scheme. Interestingly, it is 
not true that all monotone randomized CR schemes can be obtained as distributions of deterministic ones. Now the 
question is whether we can solve (LPl) restricted to monotone deterministic schemes. In general this is a non-trivial 
problem. However, the ellipsoid-based algorithm to compute c(x, $*) that we described above gives the following 
important property. In each iteration of the ellipsoid algorithm, the separation oracle uses a maximum-weight inde- 
pendent set algorithm for I to find a violating constraint; this constraint corresponds to a deterministic scheme that 
is obtained by specializing the algorithm to the given weight vector y. Therefore, if the maximum-weight indepen- 
dent set algorithm is monotone, then all the constraints generated in the ellipsoid algorithm correspond to monotone 
schemes. Since we solve the primal (LPl) only for the schemes generated by the separation oracle for the dual (DPI), 
it follows that there is an optimum solution to (LPl) that is a distribution over monotone schemes! In such a case 
c(x, $*) = c(x, $) and there is no loss in using monotone schemes. For matroids the greedy algorithm to find a 
maximum weight independent set is a monotone algorithm. Thus, for matroids, the above approach of solving (DPI) 
and (LPl) can be used to obtain a close to optimal monotone (6, c)-balanced CR scheme. It remains to determine 
the value of the optimal c and we analyze it in Section l4!4l It may be the case that there is no monotone maximum 
weight independent set algorithm for some given I, say the intersection of two matroids. In that case we can use an 
approximate montone algorithm instead. 

We summarize the above discussions in the following theorem. 

Theorem 4.3. There is a {b, c)-balanced CR scheme for Px iffEf^^^x.) [maxgcfl.SGi SigS 2^*] — ^X); 2^*^*/'"" '^'^ 
X G bPx and y > 0. Moreover, if there is a polynomial-time deterministic algorithm to find a maximum weight 
independent set in X, then for any b and e > 0, there is a randomized efficiently implementable (6, c* — e)-balanced 
CR scheme for Px where c* is the smallest value of c such that there is a (b,c)-balanced CR scheme for Px; the 
running time is polynomial in the input size and 1 / e. In addition, if the maximum-weight independent set algorithm is 
monotone, the resulting CR scheme is monotone. 

Before leveraging the above theorem to design close to optimal CR schemes for matroids, we highlight an inter- 
esting connection between CR schemes and a concept known as correlation gap. This connection is a further insight 
that we gain through the linear programs (LPl) and (DPI). 



4.3 Connection to correlation gap 

In this section we highlight a close connection between CR schemes and a concept known as correlation gap |2]. The 
correlation gap is a measure of how much the expected value of a function with respect to some random input can vary, 
if only the marginal probabilities of the input are fixed. 

Definition 4.4. For a set function f : 2^ ^ M+, the correlation gap is defined as 

xe[o,i]" /+(x) 

where i?(x) is a random set independently containing each element i with probability Xi, and 
/+(x) = max{y^ asf{S) : ^ aglg = x, ^ as = 1, as > 0} 
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is the maximum possible expectation of f over distributions with expectation x. Furthermore, for a class of functions 
C, the correlation gap is defined by k{C) — infygc 

In other words, the correlation gap is the worst-case ratio between the multilinear extension F{x) = E[/(i?(x))] 
and the concave closure /^(x). We remark that we define the correlation gap as a number k <E [0, 1], to be in line 
with the parameter c in our notion of a {b, c)-balanced CR scheme (the higher the better). The definition in JJ) uses 
the inverse ratio. 

The relationship between CR schemes and correlation gap arises as follows. 

Definition 4.5. Fori C 2^, we define the correlation gap as k{T) — infxePx.y>o ^^.^. E[maxgcj?,sgi X^igs 2^*]' 
where R = i?(x) contains element i independently with probability Xi. 

The reason we call this quantity a correlation gap (considering Definition 14.41 ). is that this quantity is equal to the 
correlation gap of the weighted rank function corresponding to I (see Lemma|4j7]below). 

Tlieorem 4.6. The correlation gap of I is equal to the maximum c such that X admits a c-balanced CR scheme. 

Proof. The correlation gap of X is equal to the optimum value of (DPI). By LP duality, this is equal to the optimum 
of the primal (LPl), which is the best value of c for which there is a c-balanced CR scheme. □ 

The following lemma shows a close connection between the correlation gap of a solution set X and the correlation 
gap of the respective rank function. More precisely, the correlation gap of X corresponds to the worst (i.e. smallest) 
correlation gap of the respective rank function over all weight vectors. 

Lemma 4.7. For X C 2^ ayid weight vector y > 0, let ry{R) — niaxscR.seiJ^iesVi denote the associated 
weighted rank function. Then k{X) = infy>o ^(ry). 

Proof. Using the notation ry{R) for the weighted rank function with weights y, the correlation gap of X can be 
rewritten as k{X) = infxePi,y>o — where i?(x) contains elements independently with probabilities x^. We 
first observe that for any x G Px, we have ^^(x) = J^i ^iVi- Hence, let x e Pj, and consider a convex combination 
X = X^sei Q^slS' ^cts ^, cts > with r+ (x) ~ X^sei ^sy{S). Since the weighted rank function of a feasible 
set S £ X is simply its weight we obtain 

'^y = E ^sy{S) = y • E "■S'ls = y • X = ^x,y„ 

sex sex % 

as claimed. Therefore, 

inf m^m. 

xePi.y>o 22i^^y^ xePx,y>o r^(x) 
To prove the claim it remains to show that 

.^^^ E[ry(i?(x))] _ .^^ E[ry(i?(x))] ^ 
xGPx,y>0 ry(x) xe[0,l]",y>0 Ty (x) 

Let y > 0. We will prove (|9]l by showing that for any point x G [0, 1]^ there is a point x' e Pi with x' < x 
(coordinate- wise), and satisfying (x') > r+(x). Since Vy is monotone, we then obtain E[ry(i?(x))]/rjjr(x) > 
E[ry (i?(x))]/r+(x'), showing that the infinum over x on the right-hand side of (|9]l can indeed be restricted to the 
polytope Px- Let x — '^scn '^s'^s^ X^scjv "^■S ~ ^I'^s > be a convex combination of x such that ^^(x) ~ 
J2scN (^si'yiS). For every~S C N, let I{S) C S* be a maximum weight independent set, hence ry{S) = y{I{S)). 
The point x' = J^scn Q^s1/(S) clearly satisfies x' < x, and furthermore 

r+(x') > ^ ( ^w]ry{S) ^ ^sryiS) = r+(x). 

sex \wCN,I{W)=S ) SQN 

□ 
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4.4 Contention resolution for matroids 



In this section we prove the following theorem on CR schemes for matroids. 

Theorem 4.8. For any matroid Ai — {N,I) on n elements, and x G 6 ■ Pi, there exists a {b, 
CR scheme. 

We later address monotonicity of the scheme and constructive aspects. To prove Theorem 14.81 we rely on the 
characterization formalized in Theorem l4.3l It suffices to prove for x € 6- Pj and any non-negative weight vector y > 

that E/j [max5c_R,sei X^ieS ^ '^^ieN ^iUi^ v^i^ c = ^ where R contains each i G N independently 

with probability Xi and x G 6 • Px- For a given weight vector y > on and a set 5 C iV let ry{S) denote the 
weight of a maximum weight independent set contained in S; in other words Vy is the weighted rank function of the 
matroid Ai. Restating, it remains to prove 

1 — fl — 

E[ry(i?)] > J2 y^^^' ^10) 

i<£N 

It is well-known that a simple greedy algorithm can be used to compute ry{S) (in fact an independent set S' C S 
of maximum weight with respect to y^): Start with S' = 0, consider the elements of S in non-increasing order of their 
weight Ui and add the current element i to 5" if S' + i is independent, otherwise discard i. 

To show ( fTOb . which is a general property of weighted matroid rank functions, we prove a more general result 
that holds for any nonnegative monotone submodular function. The main ingredient is the following Lemma l4!9l We 
remark that a slightly weaker lemma with a shorter proof was presented in an earlier version of this paper [il7'i . Both 
lemmas can be seen as an extension of the property that the correlation gap for monotone submodular functions is 
1 — 1/e ISl. The proof of Lemma|4j9]can be found in AppendixlAl 

Lemma 4.9. If f : 2^ — > M.^ is a monotone submodular function, F : [0, 1]^ — > R+ its multilinear extension, and 
/+ : [0, 1]^ — )• E+ its concave closure, then for any b E [0, 1] and p £ [0, 1]^, 

F(6.p)> (l-(l-^)")/+(p). 

Lemma |4!9] implies ( fTOl i. and therefore completes the proof of Theorem l4.8l by setting f = ry and b p = x. Notice 
that the multilinear extension of ry evaluated at x is E[ry (i?)]. Furthermore, ry (p) = J2ieN ViPi = X^iew Viir if P 
is in the matroid polytope. Hence we obtain ( fTOl i: 

EM«)l.(i-(i-^)>.;(P)^(i-(i-0")g».^ 

Theorem l4.3l also shows that an efficient algorithm for computing ry results in an efficiently implementable near- 
optimal CR scheme. It is well-known that a simple greedy algorithm can be used to compute ry(5) (in fact an 
independent set S" C 5 of maximum weight): Start with S' ~ 0, consider the elements of S in non-increasing order 
of their weight and add the current element i to S' if S' + i is independent, otherwise discard i. Moreover, it is easy to 
see that this algorithm is monotone — the ordering of the elements by weight does not depend on the set S and hence 
if an element i is included when evaluating ry{A) then it will be included in evaluating ry{B) for any B C A. We 
thus obtain our main result for CR schemes in the context of matroids by combining Theorem l4.3l for a choice of e 
satisfying e < with Theorem |4.8l and by using the inequality (1 — ;^)" < — j^F"i 

'"This inequality can be obtained by obsei'viiig that 1 — x + ^ < e^^ for x g [0, 1], and hence (1 — < {e~n — ^^-j.)". Let y = e~n 

and z = for simphcity. One can easily check that for these values of y and z we have {y — z)" < i/" — ny"^^z + Expanding 
the last expression and using n > 2, since the inequality is trivially true for n = 1, the desired inequality follows. 



j -balanced 
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Corollary 4.10. For any matroid Ai, and x £ b ■ Px, there is an efficiently implementable (b, ^—^ — ^-balanced and 
monotone CR scheme. 

As shown by the following theorem, the CR schemes that can be obtained according to Corollary 14. 101 are, up to 
an additive e, asymptotically optimal. 

Theorem 4.11. For any b G (0, 1], there is no {b,c) -balanced CR scheme for uniform matroids of rank one on n 
elements with c > - — . 

Proof. Let A4 ~ be the uniform matroid of rank 1 over n = |iV| elements, and consider the point x E b ■ Px 

given by Xi = b/n for i E N. Let i? be a random set containing each element i E N independently with probability 
Xi . The expected rank of R is given by 

/ 6 \ " 

E[r(i?)] = 1 -Pr[i? = 0] = 1 - n - - j . (11) 
Moreover, any (&, c)-balanced CR scheme returning a set / e I satisfies 

E[\I\]^J2Pr[^eI]>J2- = bc. (12) 

ieN ieN 

Since / is an independent subset of R we have E[r(i?)] > E[|/|], and the claim follows by (fTTI ) and ( fT2b . □ 



A simple (6, 1 — 5)-balanced CR scheme: Here we describe a sub-optimal (6, 1 — 6)-balanced CR scheme for matroid 
polytopes. Its advantage is that it is deterministic, simpler and computationally less expensive than the optimal scheme 
that requires solving a linear program. Moreover, Lemma 14.121 that is at the heart of the scheme, is of independent 
interest and may find other applications. A similar lemma was independently shown in ifTTl (prior to our work but in 
a different context). Let M = {N,I) be a matroid. For S C N recall that r{S) is the rank of S in Ai. The span of a 
set S denoted by span(S') is the set of all elements i G N such that r{S + i) = r{S). 

Lemma 4.12. If A4 — {N,X) is a matroid, x G P{A4), b e [0, 1] and R a random set such that Pr[i E R] = bxi, 
then there is an element iq such that Pr[io G span{R)] < b. 

Proof Let r{S) = max{|/| : / C S* & / e 1} denote the rank function of matroid M = (N.T). Since x e P{M), 
it satisfies the rank constraints x[S) < r{S). For S = span(i?), we get 

x(span(i?)) < r(span(i?)) = r{R) < \R\. 

Recall that Ris a random set where Pr[i E R] = bxi. We take the expectation on both sides: 

E[a:(span(i?))] = Pr[i G span(i?)], andE[|i?|] = ^Pr[i E R] = b'^Xi. 

i i i 

Therefore, 

Xi Pr[i E span(i?)] < b ^ Xi. 

ieN ieN 

This implies that there must be an element iq such that Pr[io E span(i?)] <b. □ 

We remark that the inequality x,; Pr[z E span(i?)] < E[|i?|] has an interesting interpretation: If x € P{M), 
we sample R with probabilities Xi, then let S = span(i?) and sample again S' C S with probabilities Xi, then 
E[|S"|] < E[|i?|]. We do not use this in the following, though. 

Theorem 4.13. For any matroid Ai and any b E [0, 1], there is a deterministic {b, 1 — b)-balanced CR scheme. 
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Proof. Let x S P{M.) and sample R with probabilities hxi. We define an ordering of elements as follows. By 
Lemma 14.121 there is an element ig such that Pr[i G span(i?)] < h. We place at the end of the order Then, since x 
restricted to \ {zq} is in the matroid polytope of M.\ {io}, we can recursively find an ordering by the same rule. If 
the elements are labeled 1,2,..., | in this order, we obtain that Pr[i € span(i? fl [i])] < b for every i. In fact, we 
are interested in the event that i is in the span of the preceding elements, RO [i — 1]. This is a subset of i? n [i], and 
hence 

Pr[i e span(i? n [i - 1])] < Pr[z £ span(iT', n [i])] < b. 
The CR scheme is as follows: 

• Sample R with probabilities bxi. 

• For each element i, if i E R \ span(i? n[i — 1]), then include it in /. 

Obviously, r{I D [i]) = r{I n [i — 1]) + 1 whenever i E I,so r{I) = |/| and / is an independent set. 

To bound the probability of appearance of i, observe that the appearance of elements in [i — 1] is independent of 
the appearance of i itself, and hence the events i G R and i ^ span(i? fl [i — 1]) are independent. As we argued, 
Pr[i G span(_R n [i — 1])] < b. We conclude: 

Fr[i e I\ie R]= Pr[i ^ span(i? n[i- 1])] > I - b. 

□ 

To implement the scheme we need to make Lemma l4.12l algorithmic. We can accomplish it by random sampling. 
Fix an element i. Pick a random set R and check if i € span(i?); repeat sufficiently many times to obtain an accurate 
estimate of Pr[i <E span(i?)]. We note that although the scheme itself is deterministic once we find an ordering of the 
elements, the construction of the ordering is randomized due to the estimation of Pr[i G span(i?)] via sampling. 

4.5 Contention resolution for knapsacks 

Here we sketch a contention resolution scheme for knapsack constraints. This essentially follows from known tech- 
niques; we remark that Kulik, Shachnai and Tamir ll34l showed how to round a fractional solution to the problem 
max{i^(x) : x e P} for any constant number of knapsack constraints and any non-negative submodular function, 
while losing a (1 — e) factor for an arbitrarily small £ > 0. Our goal is to show that these techniques can be implemented 
in a black-box fashion and integrated in our framework. We prove the following lemma. 

Lemma 4.14. For any d,e > and a knapsack constraint J-' = {S : X]i£S ^ 1} such that ai < 5 for all i, there is 
a monotone (1 — £, 1 — e^^^'*^ / ^^)-balanced contention resolution scheme. 

Proof. The CR scheme works as follows: given x e (1 — e) • Pjr = {x > : ^ aiXi < 1 — e}, we sample R 
with probabilities Xi. Then we set I = R if J^ieR^i — ^ ^^'^ I — % otherwise. This is obviously a monotone 
scheme. To prove the balance guarantee, we use a Chernoff bound: Since x S (1 — e)Pjr, we have ji ~ ^VllieR '^^l ~ 
(1 — e) o-iXi < 1 — £■ If M ^ 1/2. then £ < 1/2 and by the Chernoff bound (using ai G [0, 5\) 



Pr 


> 1 


< Pr 


^fli > (1 + £)m 















If /i < 1/2, then again by the Chernoff bound. 



Pr 


^a. > 1 


< Pr 


a» > 2^ 






.ieR 




jeR 





□ 

This contention resolution scheme is directly applicable only if the item sizes are relatively small compared to the 
knapsack capacity. However, standard enumeration tricks allow us to apply this scheme to general instances as well. 
This can be done for any constant number of knapsack constraints. We formulate this as follows. 
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Corollary 4.15. For any constant fc > 1 and e > 0, there is a constant uq (that depends only on e) such that for any 
submodularmaximization instance involving k knapsack constraints (and possibly other constraints), there is a set T 
of at most uq elements and a residual instance on the remaining elements such that 

• Any a-approximate solution to the residual instance together with T is an a{l — ke)-approximate solution to 
the original instance. 

• In the residual instance, each knapsack constraint admits a (1 — e, 1 — e)-balancedCK scheme. 

Proof. Given e > 0, let (5 = 0{e^ / log{l/£)) and no = l/{5e). Select T greedily from the optimal solution, by 
picking elements as long as their marginal contribution is at least SeOPT; note that |r| < uq. We define the residual 
instance so that S is feasible in the residual instance iff 5'UT is feasible in the original instance. The objective function 
in the new instance is g defined by setting g{S) = f{S U T) for each set 5 C N \T; note that <? is a non-negative 
submodular function if / is. In addition, in the residual instance we remove all elements whose size for some knapsack 
constraint is more than 6 ■ r where r is the residual capacity. The number of such elements in a knapsack can be at 
most 1/5 and hence they can contribute at most eOPT; we forgo this value for each knapsack. We obtain a residual 
instance where all sizes are at most 6 with the capacities normalized to 1. By Lemma |4. 141 each knapsack admits a 
(1 - e, 1 - e-"(^'/'')) = (1 - £, 1 - £)-balanced CRS. □ 

An advantage of this black box approach is that knapsack constraints can be combined arbitrarily with other types 
of constraints. They do not affect the approximation ratio significantly. However, the enumeration stage affects the 
running time by an 0(n"") factor 

4.6 Sparse packing systems 

We now consider packing constraints of the type < b, where x e {0,1}'^ is the indicator vector of a solution. 
We can assume without loss of generality that the right-hand side is b — 1. We say that the system is fc-sparse, if 
each column of A has at most k nonzero entries (i.e., each element participates in at most k linear constraints). The 
approximation algorithms in ||4] can be seen to give a contention resolution scheme for A:-sparse packing systems. 

CR scheme for fc-sparse packing systems: 

• We say that element j participates in constraint i, if a^j > 0. We call an element j big for this constraint, if 
aij > 1 /2. Otherwise we call element j small for this constraint. 

• Sample i? with probabilities Xi. 

• For each constraint i: if there is exactly one big element in R that participates in i, mark all the small elements 
in R for this constraint for deletion; otherwise check whether J^jen'^ij > 1 ^^'^ if mark all elements 
participating in i for deletion. 

• Define / to be i? minus the elements marked for deletion. 

Based on the analysis in ID, we obtain the following. 

Lemma 4.16. For any b € (0, the above is a monotone (b, 1 — 2kb)-balanced CR scheme for k-sparse packing 
systems. 

Proof. Let x = 6 • y with y e [0, 1]^, Ay < 1. Consider a fixed element j*. It appears in R with probability Xj*. 
We analyze the probability that it is removed due to some constraint where it participates. First, note that whether big 
or small, element j* cannot be removed due to a constraint i if the remaining elements have size less than 1/2, i.e. if 
J2jeR\{j'} '^ij < 1/2- This is because in this case, there is no other big element participating in I, and element j * is 
either big in which case it survives, or it is small and then J2j£R ^ij — 1' i-^- '^he constraint is satisfied. 

Thus it remains to analyze the event 'Yl,jeR\{j'} '^y — V^- Note that this is independent of item j* appearing in 
R. By the feasibility of ^x, EEjg_R\(j.} Ajj] = Hj^j* < ^- By Markov's inequality, Pr[Ejei?\{j*} «ii > 

1/2] < 2b. So an element is removed with probability at most 26 for each constraint where it participates. By the 
union bound, it is removed by probability at most 2fc6. 

□ 
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Recall the notion of width for a packing system: W ~ L max-^ ■ a -I ' where aij are the entries of the packing matrix 
(recall that we normalize the right-hand side to be b = 1). Assuming that W > 2, one can use a simpler CR scheme 
and improve the parameters. 

CR scheme for fc-sparse packing systems of width W: 

• Sample R with probabilities Xi. 

• For each constraint i for which J2j£R ^ij > 1' mark all elements participating in i for deletion. 

• Define / to be i? minus the elements marked for deletion. 

Lemma 4.17. Foranyh G (0, the above is a monotone [b, l — k{2eb)^~^)-balancedCR scheme for any k-sparse 
system of packing constraints of width W > 2. 

Proof. Again, let x = 6y with y e [0, 1]^, Ay < 1. Let us consider an element j' and a constraint J^ies — ^ 
that j' participates in. If we condition on j' being present in R, we have /i^ = ^[J2jeR\{j'} I ^' i?] = 
Tlij^j' (^ij^ij ^ ^- the width property, we have aij' < 1/W < 1/2. We use the Chernoff bound for [0, 1] random 
variables, Pr[X > (1 + < (eV(l + < (e/(l + S))'^^+^'>'', with 1 + 5 = {1 - aij,)/p, > 1/(26). Since 

our random variables are bounded by [0, max a^], we obtain by scaling 

Pr 

/ \ (l+i5)/ii/ max aij 

Therefore, each element is removed with probability at most (2e5)^^^ for each constraint where it participates. □ 

4.7 UFP in paths and trees 

We consider the following routing/packing problem. Let T = {V, E) be a capacitated tree with denoting the 
capacity of edge e E E. We are given k node pairs siti, . . . , Sktk with pair i having a non-negative demand d^; we 
assume (i,„ax = max,; d,; < Wmin = miiie Ue (the no-bottleneck assumption). Let N = {1, . . . ,k}. We say that 
S C N is routable if for each i E S a demand di is routed along the unique path from Si to ti, and the total flow on any 
edge e is at most Uf,. Previously a constant factor approximation has been given for the problem of finding a maximum 
weight subset of routable demands lfT4l : the problem is APX-hard even for unit-demands and unit- weights lIZTl . Let 
I = {5* C iV I 5 is routable}. Here we consider maxsex f{S) for a non-negative submodular function /. A natural 
(packing) LP relaxation for Px has a variable Xi G [0, 1] for each pair i and a constraint J2i-eeQ — for each 
edge e where Qi is the unique Si-ti path in T. 

CR scheme for unit-demands: 

• Root T arbitrarily. Let depth of pair Siti be the depth of the least common ancestor of Si and ti in T. 

• Sample R with probabilities bxi. Let / = 0. 

• Consider pairs in R in increasing order of depth. 

• Add i to / if / U {i} is routable, otherwise reject i. 

• Output/. 

The techniques in (|9][14l give the following lemma. 
Lemma 4.18. For any b & (0, ^) the above is a (6, 1 — Yz^)-balanced CR scheme. 



J2 « u > 1 1 / e i?. 



jeR 
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Proof. Let x G 6 • Px- Consider a fixed pair i* and let v be the least common ancestor of Si* and ti* in the rooted 
tree T; note that v could be one of si* or t;. . Let P be the unique path in T from v to s^* and P' be the path 
from V to . Without loss of generality assume that v ^ Si. and hence P is non-empty. We wish to upper bound 
Pr[i* ^ I \ i* G R], that is, the probability that i* is rejected conditioned on it being included in the random set 
R. The reason that i* gets rejected is that at least one edge e S P U P' is already full from the pairs that have been 
accepted into / prior to considering i*. We upper bound the probability of this event happening for some edge in P 
and use a symmetric argument for P'. 

Let ei , 62 , . . . , eft, be the edges in P from v to s^. . Let £j be the event that i* gets rejected at ej, that is, the capacity 
of Cj is full when i* is considered for addition to /. Note that these events are correlated. We claim the following; 
if j > h and > Ue,^ then £j happens only if Eh happens. The reason for this is the order in which the pairs in 
R are considered for insertion. When i* is considered, the only pairs inserted in / prior to it are those whose depth 
is no larger, and hence the total capacity used on an edge decreases as we traverse the path P from v to Sj. Thus, to 
analyze the probability of rejection it suffices to consider a subsequence of ei, 62, . . . , eft starting with ei such that the 
capacity of the next edge in the sequence is strictly smaller than the previously added one. For notational simplicity 
we will therefore assume that > Ue^ > . . . > > 1- 

Let Sj^{i^i*\e^Qi] be the set of pairs other than i* that contain e in their path Qi. Let be the event that 
\R n S'j I > • It is easy to see that Pr[£'j] < Pr[£'j]. Since -jx is a feasible solution to the LP relaxation we have 
X^ies ^ bu^.. Letting be the event that i E R, and X = X^ies -^i^ have Pr[£'j] = Pi-[X > UgJ. Since 
X is the sum of independent [0, 1] random variables Xi, and has expectation bue^, we obtain by standard Chernoff 
bounds: 

Pr[£;] = PliX > < (eV(l + Sy+'r < + 
where jj, = hUf.. and (5=1/5—1. Hence, Pr[£'j] < (eb)"'^ . Taking the union bound over all edges in the path, the 

probability of rejection of i* on some edge in P is at most Xj=i(^^) "°^ — Xfci(^^)^ ~ T^TE' where the inequality 
is due to the fact that the edge capacities are strictly decreasing and lower bounded by 1, and the equality is due to the 
fact that eb < 1 (recall that b G (0, -^))- By a union bound over P and P' we have that the probability of i* being 
rejected conditioned on it being in R is at most j^^. □ 

CR scheme for general demands: A CR scheme for general demands can be obtained as follows. The linear program 
Px is a packing LP of the form Ax < b, x € [0, 1] where A is column-restricted (all the non-zero values in a column 
have the same value). For such column-restricted packing integer programs (CPIPs), when demands satisfy the no- 
bottleneck assumption, one can use grouping and scaling techniques first suggested by Kolliopoulos and Stein 131)1 
(see also lfT4l ) to show that the integrality gap for a CPIP with matrix A is at most a fixed constant factor worse than 
that of the underlying 0-1 matrix A' (obtained from A by placing a 1 in each non-zero entry). Note that in the context of 
the UFP problem, the matrix A corresponds to the problem with arbitrary demands while the matrix A' corresponds to 
the one with unit-demands. One can use the same grouping and scaling techniques to show that a monotone (6, 1 — b')- 
balanced CR scheme for A' can be used to obtain a monotone (6/6, (1 — 5')/2)-balanced CR scheme for A. We give 
a proof in Section l4~8l see Theorem |4.19l Using this general conversion theorem and Lemma 14.181 one can obtain a 
{b, 6')-balanced CR scheme for UFP in trees for some sufficiently small but absolute constants b and b'. This suffices 
to obtain a constant factor approximation for maximizing a non-negative submodular function of routable requests 
in a capacitated tree. However, the {b/6, (1 — 6')/2)-balanced CR scheme does not allow composition with other 
constraints via Lemma [TT6l since (1 — 6')/2 does not tend to zero even if b' does. However, Theorem l4.19l gives a more 
refined statement that is helpful in applications in light of Remark fTTSl 

Without the no-bottleneck assumption, the linear program has an il{n) integrality gap even for UFP on paths JO)- 
One can still apply the grouping and scaling techniques without the no-bottleneck assumption under a mild restriction; 
we refer the reader to [[T3l . 

4.8 Column-restricted packing constraints 

Here we consider CR schemes for CPIPs. We follow the notation from |fT4l . Let A be an arbitrary m x n {0, 1}- 
matrix, and d be an n-element non-negative vector with dj denoting the jth entry in d. Let A[d] denote the matrix 
obtained by multiplying every entry of column j in A by dj. A CPIP is a problem of the form maxwx, subject to 
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A[d]x < b, X G {0, 1}". Note that all non-zero entries in A[d] for any given column have the same value and hence 
the name column-restricted. Here we are interested in submodular objective functions and the goal is obtain a CR 
scheme for the polytope Pj induced by the relaxation yl[(i]x < b, x G [0, 1]" . Instead of focusing on the polytope for 
a given d and b, we consider the class of polytopes induced by all d, b. 

Theorem 4.19. Suppose there is a monotone (/3, 1 — CR scheme for the polytope Ax. < b, a; £ [0, 1]" for every 
b g Z-|_ where A is {0, \}-matrix. Then there is a monotone (/3/6, (1 — P')/2)-balancedCR scheme for the polytope 
A[d\x. < b, a; S [0, 1]" for all d, b such that dmax = maxj dj < 6min = niin-/ bj. Moreover there is a monotone 
(/3/6, 1 — j3')-balanced CR scheme if all dj < &min/3 or if all dj > femin/3. 

We sketch the proof of the above theorem which follows the grouping and scaling ideas previously used in ll3n[T4l . 
We have chosen some specific constants in the theorem for simplicity. One can obtain some generahzations and 
variations of the above theorem via the same ideas. 

Let = {1, . . . , n} be a ground set corresponding to the columns. Given d, for integer /i > we let Nh = {j E 
N \ dj dmax/3'']}. We think of the columns in Nq as large and the rest as small. The overall idea is 

to focus either on the large demands or the small demands. Moreover, we will see that small demands can be treated 
independently within each group Nh- Let z be a feasible solution to the system A[c?]x < b, x e [0, 1]". For integer 
/i > we let z'' denote the vector obtained from z as follows: z^ = ^j/^^ if j G and z^ = otherwise. The 
vector z'' restricts the solution z to elements in Nh and scales it down by a small constant factor We also define a 
corresponding vector b'' where 6f = rj4iz''] for each row i. We have the following lemma which is a restatement of 
corresponding statements from ll3T|[T4l . 

Lemma 4.20. For h>0, let e {0, 1}" be a feasible integral solution to Ax. < b'\ x e [0, 1]" such that yj = if 
tI] = 0. Then A{d\y^ < h andJ2h>i A[d]y^ < b. 

Proof. Fix some h and consider the i-th row of y4[(i]y'* which is equal to J^jeNh '^j^ijUi- upper bound this 
quantity as follows: 

< 
< 



< 



< 

For ft, = we need a slight variant of the above where we replace b^ by max{l, 2 ^^^^^ Aijz'^} since \a] < 
max{l, 2a}. Then we obtain that 

djAijU^l < max{dmax, ^ A^jdjZj} < bi, 
since dmax < ^'min and z is feasible. Thus v4[d]y° < b. 



Aijy^ (from definition of 7V/i) 



3'' 

^b'l (feasibility of y'') 

d, 



3 



3 



P 1 + ^ A^jz'l (definition of b'' and using \a\<l + a) 

^ j 1 + ^ Aij Zj /6 (from definition of z'') 



^max 



a'' 2 



i A.jdjZj {dj > <ax/3''+i for j e Nh). 
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For the second part of the claim, consider a row i 

< 

< 

< 

The penultimate inequality is from the feasibility of z, and the last inequality is from the assumption that (i,„ax < 

bmin- □ 

With the above claim in place we can describe the CR scheme claimed in the theorem. Let z be a feasible solution 
and let z'' for /i > be constructed from z as described above. 

CR scheme: 

• For each h > Q independently run the (/3, 1 — /3')-balanced CR scheme for the polytope Ax. < b'*, a; G [0, 1]" 
with fractional solution z'' to obtain integral vectors y'', h>Q. 

• With probability 1/2 output y*', otherwise output X]/i>i y''- 

We claim that the above scheme is a monotone (/3/6, (1 — /3')/2)-balanced CR scheme. Note that we use the 
unit-demand scheme in a black-box fashion. First, we observe via Lemma 14.201 that the output of the scheme is a 
feasible integral solution. An alternative description of the scheme is as follows. We are given a point x = ^z with 
z G [0, 1]", Az < b. Obtain a set i? C by independently sampling each j G N with probability Xj ~ /3/6 • zj. 
Let Rh = RD Nh- For each h obtain Ih C Ri^ as the output of the scheme for Ay < b'\ y G [0, 1]" given the 
random set Rh- With probability 1/2 output I = Iq otherwise output / — Uh>ilh. For j G Nh we have that 
Pr[j G Iji I j G Rh] > 1 — /3'. Further, Pr[j G / | j G Ih] = 1/2 by the choice of the algorithm in the second step. 
Therefore Pr[j G / | j G i?] > (1 — It is easy to verify the scheme is monotone. 

Further, if we only have large demands or only small demands then the second step is not necessary and hence we 
obtain a (/3/6, (1 — /?')) -balanced CR scheme. 

Acknowledgments: We thank Mohit Singh for helpful discussions on contention resolution schemes for matroids, 
and Shipra Agrawal for discussions concerning the correlation gap. We thank two anonymous reviewers for their 
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A Missing proofs of Section |4] 
Proof of Theorem 

Proof. As observed in the proof of Theorem (II. 3K it suffices to show dTJi (assuming an arbitrary ordering of the 
elements = {1, . . . , n}). Let us take the expectation in two steps, first over / conditioned on R, and then over R: 

E[/(/nH)-/(/n[z-l])] > ER[EI[UeIfRn[^-l]i^)\R]] 

= ER[Pr[i e I \ R] fRn[^-lM■ 

Note that Pr[i G / | i?] can be nonzero only if i e i?, therefore we can restrict our attention to this event: 

E[/(/n H) - /(/n [i - 1])] > Pr[i e R]-E[Pr[i e / | R]fBn[.-i]{i) \^eR]. 

On the product space associated with the distribution of R conditioned oni G R, both Pr[i G / | i?] and fRn[i-i] (*) 
are non-increasing functions, due to / being monotone with respect to R, and / being submodular. Therefore, the 
FKG inequality (see ||3|) imphes that 

EfliPr[z e / I R]fRnl^-l]{i) \ i e R] > Efl[Pr[z e I \ R] \ i e R] ■ Efl[/fl^n[.-i] (*) 1*6^] 

= Fi[ieI\ieR]-E[fRn[^-iM- 

since the marginal value /_Rn[i-i] (*) does not depend oni £ R. By the (6, c)-balanced property, Pr[i E I \ i E R] > c; 
in addition, / is either monotone or we assume that Pr[i G I \ i E R] ~ c. In both cases, Pi[i E I \ i G 
R] ■ E[/i?n[j-i](«)] > c • E[/ijn[i-i](«)]- We summarize: 

E[/(/nH)-/(/n[i-i])] > Piii e R] ■ cE[fRn[^-lM 

= cE[/(i?n W)-/(i?n[z-i])]. 

Therefore, 

n n 

E[/(/)] = /(0) + E n M) - /(/ n [z - 1])] > /(0) + c ^ E[f{R n W) - f{R n [^ - l])] > cE[/(i?)]. 

i=l i=l 

□ 
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Proof of Lemma 14.91 

To prove Lemma|4j9]we use a property of submodular functions presented in ll47l . which is stated below as Lemma lATI 

Lemma A.l ( II47I ). Let f : 2^ — > K_|- be a monotone submodular function, and Ai, . . . , Am C N. For each j G [rn] 
independently, sample a random subset Aj(qj) which contains each element of Aj with probability qj. Let J be a 
random subset of [m] containing each element j € [to] independently with probability qj. Then 




E[/(Ai(gi)U---UA™((7„))] >E 

Lemma lA!2] below is a generahzation of Lemma 4.2 in We follow the same proof technique as used in pTl . 
The lemma contains two statements. The first is a simpler statement that may be of independent interest. The second, 
which can be seen to be a slightly stronger version of the first statement, turns out to imply Lemma (14.9b . as we will 
show in the following. 

Lemma A.2. Let f : 2^ — > R_|_ be a monotone submodular function, and Ai, . . . , Am C N. For each j S [to] 
independently sample a random subset Aj{qj) which contains each element of Aj with probability qj. Let q = 
j=i Qj- Then 

E[/(Ai(gi) U • • • U Am{qra))] > - (l - (l - ^)") Y.^of{Aj). 



Furthermore for m > 2 and any s, t € [m] with s t, 
E[/(Ai(gi)U---U A„(g„))] 

1 q-Qsqt 



m — 1 ^ 



> ^ (^1 - (1 - ^ j j l^-'^-^'^* --{•^(^^)' ^(^*)> + g '^^/(^^■) 

Proof. Observe that the first statement is a consequence of the second one: it suffices to add an arbitrary additional 
set Am+i with probability gm+i = to the family of sets and invoke the second part of the lemma with s = 1 and 
t = TO + 1. Hence, we only prove the second part of the lemma. 

By Lemma IA. II it suffices to estimate E [f{\Jj^jAj)], where J is a random subset of [to] containing element 
j G [to] independently of the others with probability qj. Assume f{Ai) > • • • > f{Am) and without loss of 
generality we assume t > s. We define for k E [to], 

Jk = {I^ [m] I min(/) = k}. 

By monotonicity of /, we have /(Ujg jAj) > f{Ak) if J G jTfc. Hence, 

m m i ~ 1 

nfi^jejA,)] < ^Pr[J G J,]f{Aj) = 11(1 - 1^)- 

]=i j=i i=i 

Thus, it suffices to prove 

m i-1 1 / / — \m-l\ / m \ 

E/(^.)%n(l-'Z.)>^^(^l-(l-^;^j j i-qsqJiA,)+Y.q,fiA,)\. (13) 

Since the above inequality is linear in the parameters f{Aj), it suffices to prove it for the special case f{Ai) = ■ ■ ■ — 
f{Ar) = 1 and /(A^+i) = • • • = f{A„i) = (A general decreasing sequence of f{Aj) can be obtained as a positive 
linear combination of such special cases.) Hence, it remains to prove 
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where lr>t is equal to 1 if r > t and otherwise. To prove (fl4] i we distinguish two cases depending on whether r < t 
OT r > t. 

Case r < t: Expanding the left-hand side of ( fT4] i. we obtain 

r j-l r / 1 ' 

E'^^ 11(1 -qi)^l- 11(1 -q,)>l - [l--Y.1^ 
3=1 e=i j=i \ 3=1 

using the arithmetic-geometric mean inequality. Finally, using concavity of (j)r{x) = 1 — (1 — and 0^(0) = 0, we 
get 



> 0m-i(g - qsqt) 



TJ'j = l 13 



q - qsqt \ \ m-i 



q - qsqt 

m — 1\ r 



3 = 1 



where the last inequality follows from the fact (t>r{x) is decreasing in r and by using r < m — 1. Notice that we used 
the fact X]j=i qj — q ^ qsqt for the first inequality in the reasoning above, which holds since r < t and therefore 

qj <q-qt <q- qsqt- 

Case r > t: As in the previous case we start by expanding the left-hand side of [14] This time we bundle the two 
terms (1 — qs) and (1 — qt) when applying the arithmetic-geometric mean inequality. 

r j-1 r 

J2qjl[il-qi) = l-l[{l-q,)^l-{l-qs){l-qt) [] " 

3=1 i=l 3 = 1 je[r] 

3t{sA 

r-1 



>^-U-^[-qsqt + Y.i3 



3 = 1 

The remaining part of the proof is similar to the previous case. 

Ill ^ \\ ' , I ^ \ . , ,-qsqt + EUii 
1 - 1 - ^— Y -qsqt + z^qj \ \ = ^r-i I -qsqt + 2^ 9^ > <Pr-i(g - qsqt) 



3 = 1 II \ 3 = 1 

-qsqt + Ef=i qj 



q - qsqt 



> (pm^iiq - qsqt)- 



^3 = 

q - qsqt 



3 = 1 



again using concavity of (t)r-i and the fact that (f>r-i{q — qsqt) is decreasing in r. This finishes the proof of (fl4] i and 
thus completes the proof of the lemma. □ 



Leveraging Lemma lA. 21 we are now ready to prove Lemma 1479] We recall that for a nonnegative submodular 
function / : 2^ — >• M+ and p e [0, 1]^, its concave closure /+ is defined by 

/+(p) = maxJ E«s/(5)|as>0V5C7V, ^^5 = 1, ^ «s = Vz e iV 

I SCAT SCN SCN,i<£S 
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Proof of Lemma WM Consider a basic solution {aj, A 



to the linear program that defines f~^{p), i.e., /^(p) 



ET=i ^jfi^jl with A, C N, aj > for j e [m], J2T=i = 1 and ^ 



j£lm],ieAj 



Pi for i £ N. Notice that 



since we chose a basic solution and the LP defining /"*" (p) only has n + 1 constraints apart from the nonnegativity 
constraints, we have m < n + 1. Let R{bp) be a random subset of N containing each element i E N independently 
with probability bpi. We distiguish two cases depending on whether m < n ot m = n + I. 

Case m < n: Consider the random set 



A= y A,{b-a,)., 



je[m] 

where Aj{baj) is a random subset of N containing each element i E N with probability baj, independently of the 
others. Notice that the distribution of A is dominated by the distribution of R{bp) since A contains each element 
i G N independently with probability 



jG[m] 



Hence F{bp) > E[f{A)], and we can use the first statement of Lemma |AT2l to obtain 

1=1 K \ 



F{bp) > E[f{A)] > 



1-1 



^&a,/(^,)> (l - (l 



b_ 

m 



/+(P), 



is decreasing in x. The proof of this case is completed by observing 



using J2]Li '^j ~ 1 and the fact that 
that (1 — (1 — -^)"^) is decreasing in m and m < n. 

Case m = 71 + 1: Since Aj C N for j E [n + 1] and | | = ?i, there must be at least one set At that is covered 
by the remaining sets, i.e.. At C Uj^[n^i]j^t^j- Furthermore let s E [n + 1] \ {<} be the index minimizing bag. We 
define probabilities qj for j E [n + 1] as follows 



baj 



1j 



if.7=t. 



We follow a similar approach as for the previous case by considering the random set 

A= \J A,{qj). 

je[m] 

Again, we first observe that A is dominated by the distribution of R{bp). For any i E N\At the analysis of the previous 
case still holds and shows Pr[i £ A] < Pr[i E R(bp)]. Consider now an element i E At. Let At, Aj^ , ■ ■ ■ , Aj^ be 
all sets in the family {Aj)j^[n+i] that contain i. By our choice of At, there is at least one other set containing i, i.e., 
r > 1. Using 

bat ^ 



we obtain 



(1 -(7t)(l - gji) = ( 1 - j (1 - ^"ji) ^ l-bat-baj,. 



Pr[z eA] = 1-{1- qt)il - " = 1 - (1 - - ba,,)Y[{l - ba,,) 



i=2 



1=2 



( 



< 1 



1 

\ ie[n+i] / 



bp,=Pv[i E Ribp)]. 
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Therefore, we again have F{bp) > E[/(A)]. Notice that q = J2"=i Ij satisfies 

n+l rt+1 

-4- 

1 — bag 



q = bat- h ^ baj = Qsqt + ^ baj = gf^qt + 6. 



We apply the second statement of Lemma lA!2] to the family {Aj{qj))j^[n^i] and use the above fact to obtain 

fl_fl_5_Mi] ] ( -g,g,min{/(^),/(AO}+ ^ 

9 - 



F{bp) > E[f{A)] > f 1 - f 1 - ) ) I -qsqt min{/(^), /(A*)} + 




" " ") ) min{/(^), /(A,)} + {qt - fea*) /(A*) + 6/+(p)) 



The claim follows by observing that qsqt ~ qt — bat- D 

B Details in constructing CR schemes via the ellipsoid algorithm 

Here we give the technical details that are involved in sampling and approximately solving (DPI) and (LPl) from 
Section I4l2l First a short reminder of the primal and dual problem. 

max c 

(LPl) ^'^'G** ^^■'t'h ^ \/i <E N 

Z^^e*- = 1 

> V(/) e 

min fi 

(DPI) '^^eN 1^.'I>V^ < M V0 G 

y, > yieN 

We start by observing that we can obtain strong estimates of qi^^ for any G $*. We assume that is given as an 
oracle and can therefore be evaluated in constant time. 

Proposition B.l. Let e $* and i Cz N. An estimate qi^^ ofqi^^ whose error is bounded by ±exi with high probability 
can be obtained in time polynomial in n and K 

Proof. We call a set S" C iV \ {i} good, if i e (j){S U {i}). We have 

qi,<p = Pr[i e 4>{R)] = Pr\i e R and R \ {i} is good] 

= Pr[i e R] ■ Pt[R \ {i} is good] = ■ Pr[i? \ {i} is good]. 

Notice that we can estimate Pr[i? \ {i} is good] up to an error of ±e with high probability by a standard Monte Carlo 
approach, where we draw samples of R\ {i}. This can be done in time polynomial in n and ^ and leads to the claimed 
estimate ^ by the above formula. □ 

We now discuss how these estimates can be used to obtain a near-optimal solution to (DPI) by employing the 
ellipsoid method with a weak separation oracle. After that we show how a near-optimal solution to (LPl) can be 
obtained. Notice that Proposition lB. 1 l ean easily be used to obtain estimates qi^^ of qi,^ that satisfy with high probability 
€ [qi^ff, — exi,qi^^]: it suffices to consider an estimate qi,^ of qi^^ that satisfies with high probability ^ G 
~ f 2^1,91,0 + fa^i] and to define g^^^ = qi^^ — |. In the following, we assume that all used estimates q^i.^ 
satisfy qi^cf> G — ea^i, qi,4>]- We can obtain this with high probability through Proposition IB . II since the ellipsoid 
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that we will apply in the following only uses a polynomial number of such estimates. Notice that these estimates are 
"pessimistic" estimates for q^ ^, i.e., replacing ^ in (LPl) by these estimates leads to a lower optimal value of the 
LP. 

Furthermore, to simplify the exposition, we will assume that for any given weight vector y € ]S.^ we can find a 
CR scheme G <1>* maximizing J^ieN li^ipVi- Th^ following discussion works also if we can only find a CR scheme 
that approximately maximizes this expression. 

To apply the ellipsoid algorithm to (DPI) we design a weak separation oracle (see Chapter 4 in [[321 ). As a reminder, 
the weak separation oracle has to provide the following guarantees. Given is a nonnegative vector y = {yi)ieN 
satisfying X^iew ^iVi ~ 1' ^'^'^ ^ value /i. The weak separation oracle has to provide either a feasible dual solution 
(y', i^l') with /i' < /i + e, or a hyperplane separating (y, fi) from all feasible dual solutions. Given y and ^, let 
be the CR scheme that maximizes J^ieN ^i,<t>yi- TlieN ^i,<t>yi — weak separation oracle returns the dual 

solution (y, fi + e). This solution has objective value /i + e as desired and is indeed feasible since for any (j>' £ 

^ qi.,^'yi < ^ qi.4>yi < ^{qi,4> + <^^i)yi = + X! ^^■•I'V^ < m + £• 

ieN ieN ieN ieN 

If J2ieN qi,4>Vi > M' our weak separation oracle returns the separating hyperplane given by the constraint (a, —1) • 
(z, v) <Q where a is a n-dimensional vector with coefficients a; = qi.^ for 1 <i < n (note that that (DPI) has n + 1 
variables corresponding to j/i, . . . , j/„ and /i). First, this hyperplane indeed cuts off the solution (y, /i). Furthermore, 
if (y'j is a feasible dual solution then it satisfies the constraint since < <li,4,'- 

ieN ieN 

where the second inequality in the above follows from feasibility of (y', fi'). Hence, we obtained a weak separation 
oracle for (DPI), and the ellipsoid method can therefore determine a feasible solution {y,fi) to (DPI) of value < 
fj,* +2e, where /i* is the value of an optimal dual solution. Note that since {y, fi) is feasible, we have ^* < ji < fi* + 2e 
(see 112). 

Let (DPT) be the linear program obtained from (DPI) by only considering constraints corresponding to CR 
schemes cj) that were used in the ellipsoid algorithm while constructing the nearly optimal solution (y,/i) of (DPI), 
which satisfies ^ < jJL* + 2e. Furthermore, we replace all terms g^.^ by their estimates 5^.^ in (DPT). Hence, the 
feasible region of (DPI') consists of all separating hyperplanes that were generated during the ellipsoid algorithm. 
Notice that (DPT) is a relaxation of (DPI) since our estimates satisfy < q^.^. Hence, the optimal value /i' of 
(DPT) satisfies ji' < ji* . The ellipsoid algorithm actually certifies the approximation quality of the generated solution 
(/i, y) by comparing against the best solution satisfying the generated constraints, i.e., 

/X < /i' + 2e. 

Let (LPT) be the dual of (DPT), and let (A', c') be an optimal solution to (LPT), which can be efficiently determined 
since (LPT) has polynomial size. We return (A', c') as the solution to (LPl). First, notice that (A', c') is feasible for 
(LPl) since < qi ^. Furthermore, 

c' = A*' > A* - 2e > - 2e = c* - 2e, 
where the two equahties follow by strong duality. 
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